Debugging Rules and Microservices
Jason Coposky, Chief Technologist
Capturing output from a microservice
All microservices must return an integer error --
capture with the errorcode() or errormsg() operators
Example:
*err = errorcode( msiHelloWorld() );
// or *err = errormsg( msiHelloWorld(), *msg );
if( 0 != *err ) {
// handle error here
}
Rules should communicate errors to clients
Utilize the fail() and failmsg() directives to pass errors back to the client.
*err = errorcode( msiHelloWorld() );
if( 0 != *err ) {
fail(*err);
// or failmsg( *err, "msiHelloWorld failed" );
}
This error will be propagated back to the client when run via irule or a Policy Enforcement Point
Debugging Rules
Still stuck with 'printf' style debugging --
writeLine( "serverLog", "I am Here" );
There is no proper debugger for the Rule Language but:
Rule Engine Plugins can make this better!
Classes of Errors - Syntax
- A missing } will cause a rule to complain about an error parsing the rule
- A missing “ will cause a rule to complain with a SYS_HEADER_READ_LEN_ERR, Operation now in progress
- A micro-service with incorrect number of arguments or misspelled name will be caught as such.
- Calling a delay or remote micro-service can result in “no packing instruction for arbitrary type”. This is resolved by putting the remote command call in a separate function.
- A rule that is over 2700 characters cannot be executed by a delay or remote command. This is fixed by using policy functions and loading the functions as a .re file.
- A Policy Enforcement Point with a syntax error will result in a SYS_AGENT_INIT_ERR
Classes of Errors - Microservice Errors
- All microservices should be installed on the ICAT as well as any resource servers - delayed rules are checked and stored in the ICAT
- The remote command assumes that the remote host will use the same port as the data grid from which the command is executed. An error is reported "USER_SOCK_CONNECT_ERR, Unknown error: 113”
- Failure in microservices when called from a remote zone appears as a “USER_SOCK_CONNECT_ERR”.
- As always check the server and rule engine logs
Performance Considerations
Microservices vs Rules
Rules - orchestration and flexibility
rules can be treated as 'Apps', stored in iRODS
Microservices - performance critical processing,
reach into external libraries
Leverage the Delayed Execution Queue
Rules or Microservices should not block iRODS clients -- asynchronous processing
push anything that will take a while onto the delayed execution queue
with the delay() directive
delay("<PLUSET>1m</PLUSET>) { }
The Hints are an XML style parameter list:
- ET - exact time to be performed
- PLUSET - offset from time of execution
- EF - exec frequency
Full description: http://wiki.irods.org/index.php/Rule_Execution_modes
Note: maximum_number_of_conncurrent_rule_engine_server_processes in
server_config.json controls how many delayed exec rules can be in flight
Consider a job scheduler
Volume and Run Time - consider as scheduler
- large amounts of processing requests
- long running tasks
Concerns
- Do not block the client longer than absolutely necessary
- Do not burden resource servers as compute nodes
Scheduled jobs can trigger subsequent rules within iRODS - chain HPC jobs
Questions?
Debugging Rules and Microservices
By iRODS Consortium
Debugging Rules and Microservices
An overview of techniques for debugging using the iRODS Rule Language
- 2,060