8. Workflow Automation



Examination of an example

Rule Language Overview


Rules - C-like scripts executed by the iRODS Rule Engine


Rule Engine - integrated language interpreter


Simple Example:


HelloWorld {

    writeLine("stdout", "Hello World");

Rule Execution

​Workflow may be automated in several ways

  • irule - initiated by a user


  • Policy Enforcement Points - initiated automatically


  • Delayed Execution Queue - initiated asynchronously


C++ plugins bound into the Rule Engine as functions


Useful for

  • leveraging specialized libraries - curl, gdal, etc
  • tasks which are computationally intensive


Invoked like  a function call

HelloWorld2 {

    msihello_world( *msg );


INPUT *msg = "my message"

OUTPUT ruleExecOut 


Packaged Microservices - installed via native packaging system ( rpm, dpkg )

  • Installed into /var/lib/irods/plugins/microservices


Hand-Built Microservices - copied by hand into the proper plugin directory

  • /var/lib/irods/plugins/microservices
  • $IRODS_HOME/plugins/microservices


Note - for Run-In-Place installations $IRODS_HOME will be wherever the system was built and deployed

Deploying Rules for Policy Enforcement Points

Directly edit and add rule code to the PEP defined in the default rule base:



Override the Policy Enforcement Point -

    include a new file within the rule base containing the desired rule code


Note - The rule base is configured in server_config.json

The Example Rule - command line demo


  • given - training_acPostProcForPut.re and msiget_image_meta are installed
  • add training_acPostProcForPut to /etc/irods/server_config.json
  • iput the training_jpg directory into iRODS
  • examine the harvested metadata with imeta

Examining the rule file - defining the PEP

Found at - /etc/irods/training_acPostProcForPut.re


First we will define the PEP for the rule definition:


    acPostProcForPut {


​This PEP takes no parameters, so does not need parenthesis 

Examining the rule file - filter file types

We only want to harvest metadata from image files - filter using an 'if' statement


Session Variable - global variables holding values about the data object in flight

$filePath - session variable holding the physical path


    if( $filePath like "*.jpg"  || $filePath like "*.jpeg"  ||
        $filePath like "*.bmp"  || $filePath like "*.tif"  ||     
        $filePath like "*.tiff" || $filePath like "*.rif"  ||
        $filePath like "*.gif"  || $filePath like "*.png"  ||    
        $filePath like "*.svg"  || $filePath like "*.xpm") { 

Examining the rule file - extract the metadata

Once we have filtered the file type

  • invoke the microservice to harvest the metadata
  • metadata is encoded as a string in the 'out variable' *meta


    msiget_image_meta($filePath, *meta);


Examining the rule - applying the metadata

The metadata encoded string is converted to an internal iRODS key-value data structure in the 'out variable' *meta_kvp


    msiString2KeyValPair(*meta, *meta_kvp);


Once we have the key-value pairs we apply them to our data object


    msiAssociateKeyValuePairsToObj(*meta_kvp, $objPath, "d");


The Data object is referenced by the session variable $objPath which is the logical iRODS path


The "d" signifies to the microservice that we are referencing a Data Object, not a Collection or Resource

Examining the rule - closing the scope

Now that metadata is applied we need to close our two scope blocks - one for the 'if' and the one for the PEP itself - 


        } # if
    } # acPostProcForPut

Examining the rule - the final product

acPostProcForPut {

    if( $filePath like "*.jpg"  || $filePath like "*.jpeg" ||
        $filePath like "*.bmp"  || $filePath like "*.tif"  ||
        $filePath like "*.tiff" || $filePath like "*.rif"  ||

        $filePath like "*.gif"  || $filePath like "*.png"  ||
        $filePath like "*.svg"  || $filePath like "*.xpm") {

            msiget_image_meta($filePath, *meta);

            msiString2KeyValPair(*meta, *meta_kvp);

            msiAssociateKeyValuePairsToObj(*meta_kvp, $objPath, "-d");

    } # if 

} # acPostProcForPut

Rule Language Overview

A short description can be found at the end of this section in the training manual


The full description can be found at -


UGM2015-Training - Workflow Automation

By jason coposky

UGM2015-Training - Workflow Automation

Introduction to Workflow Automation with example rule and metadata extraction microservice

  • 1,287
Loading comments...

More from jason coposky