8. Workflow Automation

Rules

Microservices

Examination of an example

Rule Language Overview

Rules

Rules - C-like scripts executed by the iRODS Rule Engine

 

Rule Engine - integrated language interpreter

 

Simple Example:

 

HelloWorld {

    writeLine("stdout", "Hello World");

} 

Rule Execution

​Workflow may be automated in several ways

• irule - initiated by a user

 

• Policy Enforcement Points - initiated automatically

 

• Delayed Execution Queue - initiated asynchronously

Microservices

C++ plugins bound into the Rule Engine as functions

 

Useful for

• leveraging specialized libraries - curl, gdal, etc
• tasks which are computationally intensive

Microservices

Invoked like  a function call

HelloWorld2 {

    msihello_world( *msg );

}

INPUT *msg = "my message"

OUTPUT ruleExecOut 

Installation

Packaged Microservices - installed via native packaging system ( rpm, dpkg )

• Installed into /var/lib/irods/plugins/microservices

 

Hand-Built Microservices - copied by hand into the proper plugin directory

• /var/lib/irods/plugins/microservices
• $IRODS_HOME/plugins/microservices

 

Note - for Run-In-Place installations $IRODS_HOME will be wherever the system was built and deployed

Deploying Rules for Policy Enforcement Points

Directly edit and add rule code to the PEP defined in the default rule base:

    /etc/irods/core.re 

 

Override the Policy Enforcement Point -

    include a new file within the rule base containing the desired rule code

 

Note - The rule base is configured in server_config.json

The Example Rule - command line demo

Goals

• given - training_acPostProcForPut.re and msiget_image_meta are installed
• add training_acPostProcForPut to /etc/irods/server_config.json
• iput the training_jpg directory into iRODS
• examine the harvested metadata with imeta

Examining the rule file - defining the PEP

Found at - /etc/irods/training_acPostProcForPut.re

 

First we will define the PEP for the rule definition:

 

    acPostProcForPut {

 

​This PEP takes no parameters, so does not need parenthesis 

Examining the rule file - filter file types

We only want to harvest metadata from image files - filter using an 'if' statement

 

Session Variable - global variables holding values about the data object in flight

$filePath - session variable holding the physical path

 

    if( $filePath like "*.jpg"  || $filePath like "*.jpeg"  ||
        $filePath like "*.bmp"  || $filePath like "*.tif"  ||     
        $filePath like "*.tiff" || $filePath like "*.rif"  ||
        $filePath like "*.gif"  || $filePath like "*.png"  ||    
        $filePath like "*.svg"  || $filePath like "*.xpm") { 

Examining the rule file - extract the metadata

Once we have filtered the file type

• invoke the microservice to harvest the metadata
• metadata is encoded as a string in the 'out variable' *meta

 

    msiget_image_meta($filePath, *meta);

 

Examining the rule - applying the metadata

The metadata encoded string is converted to an internal iRODS key-value data structure in the 'out variable' *meta_kvp

 

    msiString2KeyValPair(*meta, *meta_kvp);

 

Once we have the key-value pairs we apply them to our data object

 

    msiAssociateKeyValuePairsToObj(*meta_kvp, $objPath, "d");

 

The Data object is referenced by the session variable $objPath which is the logical iRODS path

 

The "d" signifies to the microservice that we are referencing a Data Object, not a Collection or Resource

Examining the rule - closing the scope

Now that metadata is applied we need to close our two scope blocks - one for the 'if' and the one for the PEP itself - 

 

        } # if
    } # acPostProcForPut

Examining the rule - the final product

acPostProcForPut {

    if( $filePath like "*.jpg"  || $filePath like "*.jpeg" ||
        $filePath like "*.bmp"  || $filePath like "*.tif"  ||
        $filePath like "*.tiff" || $filePath like "*.rif"  ||

        $filePath like "*.gif"  || $filePath like "*.png"  ||
        $filePath like "*.svg"  || $filePath like "*.xpm") {

            msiget_image_meta($filePath, *meta);

            msiString2KeyValPair(*meta, *meta_kvp);

            msiAssociateKeyValuePairsToObj(*meta_kvp, $objPath, "-d");

    } # if 

} # acPostProcForPut

Rule Language Overview

A short description can be found at the end of this section in the training manual

 

The full description can be found at -

https://docs.irods.org/4.1.2/manual/rule_language/