Microservice Plugins
June 7, 2016
User Group Meeting
Chapel Hill, NC
Jason M. Coposky
@jason_coposky
Interim Executive Director
The iRODS Plugin Architecture
iRODS 4.2 provides 7 plugin interfaces
- microservices
- resources
- authentication
- network
- database
- RPC API
- rule engine
Anatomy of a Plugin
- Built as a dynamic shared object
- provides a plugin_factory method which instantiates the plugin object
-
contains a class derived from a plugin interface base class:
- api_entry
- auth
- network
- resource
- database
- ms_table_entry
- pluggable_rule_engine<>
NOTE: microservices do not require a derived class, the factory can return an instance of ms_table_entry
Follow along in the irods_training repository
The code can be found at:
~/irods_training/advanced/irods_microservice_plugin/src/lib-microservice-example.cpp
Start with the Factory
extern "C" irods::ms_table_entry* plugin_factory() { irods::ms_table_entry* msvc = new irods::ms_table_entry(3); msvc->add_operation< msParam_t*, msParam_t*, msParam_t*, ruleExecInfo_t*>( "msiexample_microservice", std::function<int( msParam_t*, msParam_t*, msParam_t*, ruleExecInfo_t*)>(msiexample_microservice)); return msvc; }
Anatomy of the Plugin Factory
extern "C" irods::ms_table_entry* plugin_factory() { ... return msvc; }
- Must have C linkage
- Must return an irods::plugin_base derived type
- Must be named "plugin_factory"
Instantiating a microservice plugin
extern "C"
irods::ms_table_entry* plugin_factory() {
irods::ms_table_entry* msvc = new irods::ms_table_entry(3);
...
return msvc;
}
- allocate a raw pointer to irods::ms_table_entry
- pass its constructor the number of parameters not including the ruleExecInfo_t*
- Note: microservices do not use a derived class definition, unlike other plugin types
Wiring a plugin operation
...
msvc->add_operation<
msParam_t*,
msParam_t*,
msParam_t*,
ruleExecInfo_t*>(
"msiexample_microservice",
std::function<int(
msParam_t*,
msParam_t*,
msParam_t*,
ruleExecInfo_t*)>(msiexample_microservice));
...
- Template parameters are the parameters of the function operation
- First parameter is the calling name of the operation - "msiexample_microservice"
- Second parameter is a std::function wrapping the local function definition
- Takes the full signature of the function as a template parameter
- Takes the function pointer as an argument
The microservice definition
int msiexample_microservice( msParam_t* _string_param, msParam_t* _int_param, msParam_t* _double_param, ruleExecInfo_t* _rei ) { char *string_param = parseMspForStr( _string_param ); if( !string_param ) { std::cout << "null _string_param" << std::endl; return SYS_INVALID_INPUT_PARAM; } int int_param = parseMspForPosInt( _int_param ); if( int_param < 0 ) { std::cout << "invalid _int_param" << std::endl; return SYS_INVALID_INPUT_PARAM; } double double_param = 0.0; int ret = parseMspForDouble( _double_param, &double_param ); if( ret < 0 ) { std::cout << "invalid _double_param" << std::endl; return SYS_INVALID_INPUT_PARAM; } std::cout << __FUNCTION__ << " string [" << string_param << "] int [" << int_param << "] double [" << double_param << "]" << std::endl; return 0; }
The signature
int msiexample_microservice( msParam_t* _string_param, msParam_t* _int_param, msParam_t* _double_param, ruleExecInfo_t* _rei ) { ... return 0; }
- Returns an int
- Takes N (per the ctor) number of msParam_t*
- Must end in a ruleExecInfo_t*
Parameter parsing and Error Checking
... char *string_param = parseMspForStr( _string_param ); if( !string_param ) { std::cout << "null _string_param" << std::endl; return SYS_INVALID_INPUT_PARAM; } int int_param = parseMspForPosInt( _int_param ); if( int_param < 0 ) { std::cout << "invalid _int_param" << std::endl; return SYS_INVALID_INPUT_PARAM; } double double_param = 0.0; int ret = parseMspForDouble( _double_param, &double_param ); if( ret < 0 ) { std::cout << "invalid _double_param" << std::endl; return SYS_INVALID_INPUT_PARAM; } ....
- msParam_t* must be parsed into native types
- Individual parseMspFor* exist for most iRODS types
- Either the type is returned, if possible - or an error code
Highly complex application code
int msiexample_microservice( msParam_t* _string_param, msParam_t* _int_param, msParam_t* _double_param, ruleExecInfo_t* _rei ) { ... std::cout << __FUNCTION__ << " string [" << string_param << "] int [" << int_param << "] double [" << double_param << "]" << std::endl; return 0; }
Many things are possible in a microservice
- Calling server-side API endpoints - rsGenQuery
- Manipulating data at rest
- Return 0 for success
Building and Installing the Example Code
sudo apt-get -y install irods-externals-*
export PATH=/opt/irods-externals/cmake3.5.2-0/bin:/opt/irods-externals/clang3.8-0/bin:$PATH
which clang++
/opt/irods-externals/clang3.8-0/bin/clang++
which cmake
/opt/irods-externals/cmake3.5.2-0/bin/cmake
sudo apt-get install irods-dev
mkdir ~/build_msvc
cd ~/build_msvc
cmake ~/irods_training/advanced/irods_microservice_plugin
make package
sudo dpkg -i irods-microservice-example-4.2.0-ubuntu14-x86_64.deb
A rule to test the microservice
acPostProcForPut() {
microservice_example("XXXX - test string", 314, 123.4);
if("ufs_cache" == $rescName ) {
writeLine( "serverLog", "XXXX - calling delayed replication" );
delay("<PLUSET>1s</PLUSET><EF>1h DOUBLE UNTIL SUCCESS OR 6 TIMES</EF>") {
*CacheRescName = "comp_resc;ufs_cache";
msisync_to_archive("*CacheRescName", $filePath, $objPath );
}
}
}
Edit the /etc/irods/training.re rulebase
Ensure the iRODS Rule Language plugin is first
Edit /etc/irods/server_config.json
- Delete any other rule engine plugins
{
"instance_name" : "re-storage-balancing-instance",
"plugin_name" : "re-storage-balancing",
"plugin_specific_configuration" : {}
},
{
"instance_name" : "rule-engine-plugin-python-instance",
"plugin_name" : "rule-engine-plugin-python",
"plugin_specific_configuration" : {}
},
{
"instance_name": "re-irods-instance",
"plugin_name": "re-irods",
Test the microservice
iput VERSION.json
grep -dskip "XXXX -" ./log/*
./log/rodsLog.2016.06.XX:msiexample_microservice string [XXXX - test string] int [314] double [123.4]
Questions?
UGM 2016 - Microservice Plugins
By jason coposky
UGM 2016 - Microservice Plugins
- 1,898