Debugging Rules and Microservices

Jason Coposky, Chief Technologist

Capturing output from a microservice

All microservices must return an integer error --
    capture with the errorcode() or errormsg() operators



    *err = errorcode( msiHelloWorld() );

    // or *err = errormsg( msiHelloWorld(), *msg );

    if( 0 != *err ) {

        // handle error here


Rules should communicate errors to clients

Utilize the fail() and failmsg() directives to pass errors back to the client.


*err = errorcode( msiHelloWorld() );

if( 0 != *err ) {


    // or failmsg( *err, "msiHelloWorld failed" );



This error will be propagated back to the client when run via irule or a Policy Enforcement Point

Debugging Rules

Still stuck with 'printf' style debugging --

    writeLine( "serverLog", "I am Here" );


There is no proper debugger for the Rule Language but:

    Rule Engine Plugins can make this better!

Classes of Errors - Syntax

  • A missing } will cause a rule to complain about an error parsing the rule
  • A missing “ will cause a rule to complain with a SYS_HEADER_READ_LEN_ERR, Operation now in progress
  • A micro-service with incorrect number of arguments or misspelled name will be caught as such.
  • Calling a delay or remote micro-service can result in “no packing instruction for arbitrary type”.  This is resolved by putting the remote command call in a separate function.
  • A rule that is over 2700 characters cannot be executed by a delay or remote command.  This is fixed by using policy functions and loading the functions as a .re file.
  • A Policy Enforcement Point with a syntax error will result in a SYS_AGENT_INIT_ERR

Classes of Errors - Microservice Errors

  • All microservices should be installed on the ICAT as well as any resource servers - delayed rules are checked and stored in the ICAT
  • The remote command assumes that the remote host will use the same port as the data grid from which the command is executed.  An error is reported "USER_SOCK_CONNECT_ERR, Unknown error: 113”
  • Failure in microservices when called from a remote zone appears as a “USER_SOCK_CONNECT_ERR”.
  • As always check the server and rule engine logs

Performance Considerations

Microservices vs Rules


Rules - orchestration and flexibility

    rules can be treated as 'Apps', stored in iRODS


Microservices - performance critical processing,

                            reach into external libraries

Leverage the Delayed Execution Queue

Rules or Microservices should not block iRODS clients -- asynchronous processing

    push anything that will take a while onto the delayed execution queue

    with the delay() directive


delay("<PLUSET>1m</PLUSET>) {  }

The Hints are an XML style parameter list:

  • ET - exact time to be performed
  • PLUSET - offset from time of execution
  • EF - exec frequency

Full description:

Note: maximum_number_of_conncurrent_rule_engine_server_processes in 

           server_config.json controls how many delayed exec rules can be in flight

Consider a job scheduler

Volume and Run Time - consider as scheduler

  • large amounts of processing requests
  • long running tasks



  • Do not block the client longer than absolutely necessary
  • Do not burden resource servers as compute nodes


Scheduled jobs can trigger subsequent rules within iRODS - chain HPC jobs


Debugging Rules and Microservices

By jason coposky

Debugging Rules and Microservices

An overview of techniques for debugging using the iRODS Rule Language

  • 1,701
Loading comments...

More from jason coposky