Advanced Training:
Auditing
June 29 - July 2, 2026
iRODS User Group Meeting 2026
Barcelona, Spain
iRODS Audit Rule Engine Plugin - Audit via AMQP 1.0
The iRODS Audit Rule Engine Plugin uses Apache Qpid Proton (C++) to emit AMQP 1.0 messages for policy enforcement points (PEP) encountered by the iRODS server.
The AMQP message has all of the information related to that particular operation including username, filepath, filesize, etc.
Catching and analyzing these messages will allow visualization of totals and trends.
The rule engine is provided in our package repositories as irods-rule-engine-plugin-audit-amqp.
Inside an iCommand
Parsing the operation from each AMQP message lets us see the full flow of a client request through the server's code.
A sample of PEPs that might be hit by an iget:
audit_pep_database_gen_query_access_control_setup_pre audit_pep_database_gen_query_access_control_setup_post audit_pep_database_gen_query_pre audit_pep_database_get_rcs_pre audit_pep_database_get_rcs_post audit_pep_database_gen_query_post audit_pep_obj_stat_post audit_pep_network_write_body_pre audit_pep_network_write_header_pre audit_pep_network_write_header_post audit_pep_network_write_body_post audit_pep_auth_agent_start_pre audit_pep_auth_agent_start_post
Inside an iCommand
A larger list of PEPs that might be hit by an iget:
audit_pep_network_read_header_pre audit_pep_network_read_header_post audit_pep_network_read_body_pre audit_pep_network_read_body_post audit_pep_database_open_pre audit_pep_database_open_post audit_pep_exec_rule_pre audit_pep_exec_microservice_pre audit_pep_exec_microservice_post audit_pep_exec_microservice_pre audit_pep_database_gen_query_access_control_setup_pre audit_pep_database_gen_query_access_control_setup_post audit_pep_exec_microservice_post audit_pep_exec_rule_post audit_pep_database_gen_query_access_control_setup_pre audit_pep_database_gen_query_access_control_setup_post audit_pep_database_gen_query_pre audit_pep_database_get_rcs_pre audit_pep_database_get_rcs_post audit_pep_database_gen_query_post audit_pep_database_gen_query_access_control_setup_pre audit_pep_database_gen_query_access_control_setup_post audit_pep_database_gen_query_pre audit_pep_database_get_rcs_pre audit_pep_database_get_rcs_post audit_pep_database_gen_query_post audit_pep_exec_rule_pre audit_pep_exec_microservice_pre audit_pep_exec_microservice_post audit_pep_exec_rule_post audit_pep_exec_rule_pre audit_pep_exec_microservice_pre audit_pep_exec_microservice_post audit_pep_exec_rule_post audit_pep_exec_rule_pre audit_pep_exec_microservice_pre audit_pep_exec_microservice_post audit_pep_exec_rule_post audit_pep_network_write_body_pre audit_pep_network_write_header_pre audit_pep_network_write_header_post audit_pep_network_write_body_post audit_pep_network_agent_start_pre audit_pep_network_agent_start_post audit_pep_auth_agent_start_pre audit_pep_auth_agent_start_post audit_pep_network_read_header_pre audit_pep_network_read_header_post audit_pep_network_read_body_pre audit_pep_network_read_body_post
audit_pep_auth_request_pre audit_pep_auth_agent_auth_request_pre audit_pep_auth_agent_auth_request_post audit_pep_auth_request_post audit_pep_network_write_body_pre audit_pep_network_write_header_pre audit_pep_network_write_header_post audit_pep_network_write_body_post audit_pep_auth_agent_start_pre audit_pep_auth_agent_start_post audit_pep_network_read_header_pre audit_pep_network_read_header_post audit_pep_network_read_body_pre audit_pep_network_read_body_post audit_pep_auth_response_pre audit_pep_auth_agent_auth_response_pre audit_pep_database_check_auth_pre audit_pep_database_check_auth_post audit_pep_auth_agent_auth_response_post audit_pep_auth_response_post audit_pep_network_write_body_pre audit_pep_network_write_header_pre audit_pep_network_write_header_post audit_pep_network_write_body_post audit_pep_auth_agent_start_pre audit_pep_auth_agent_start_post audit_pep_network_read_header_pre audit_pep_network_read_header_post audit_pep_network_read_body_pre audit_pep_network_read_body_post audit_pep_obj_stat_pre audit_pep_database_gen_query_access_control_setup_pre audit_pep_database_gen_query_access_control_setup_post audit_pep_database_gen_query_pre audit_pep_database_get_rcs_pre audit_pep_database_get_rcs_post audit_pep_database_gen_query_post audit_pep_database_gen_query_access_control_setup_pre audit_pep_database_gen_query_access_control_setup_post audit_pep_database_gen_query_pre audit_pep_database_get_rcs_pre audit_pep_database_get_rcs_post audit_pep_database_gen_query_post audit_pep_obj_stat_post audit_pep_network_write_body_pre audit_pep_network_write_header_pre audit_pep_network_write_header_post audit_pep_network_write_body_post audit_pep_auth_agent_start_pre audit_pep_auth_agent_start_post
audit_pep_network_read_header_pre audit_pep_network_read_header_post audit_pep_network_read_body_pre audit_pep_network_read_body_post audit_pep_data_obj_get_pre audit_pep_database_gen_query_access_control_setup_pre audit_pep_database_gen_query_access_control_setup_post audit_pep_database_gen_query_pre audit_pep_database_get_rcs_pre audit_pep_database_get_rcs_post audit_pep_database_gen_query_post audit_pep_database_gen_query_access_control_setup_pre audit_pep_database_gen_query_access_control_setup_post audit_pep_database_gen_query_pre audit_pep_database_get_rcs_pre audit_pep_database_get_rcs_post audit_pep_database_gen_query_post audit_pep_resource_resolve_hierarchy_pre audit_pep_resource_resolve_hierarchy_post audit_pep_exec_rule_pre audit_pep_exec_microservice_pre audit_pep_exec_microservice_post audit_pep_exec_rule_post audit_pep_resource_open_pre audit_pep_resource_open_post audit_pep_resource_read_pre audit_pep_resource_read_post audit_pep_resource_close_pre audit_pep_resource_close_post audit_pep_exec_rule_pre audit_pep_exec_microservice_pre audit_pep_exec_microservice_post audit_pep_exec_rule_post audit_pep_data_obj_get_post audit_pep_network_write_body_pre audit_pep_network_write_header_pre audit_pep_network_write_header_post audit_pep_network_write_body_post audit_pep_auth_agent_start_pre audit_pep_auth_agent_start_post audit_pep_network_read_header_pre audit_pep_network_read_header_post audit_pep_network_read_body_pre audit_pep_network_read_body_post audit_pep_network_agent_stop_pre audit_pep_network_agent_stop_post audit_pep_database_close_pre audit_pep_database_close_post
Inside an iCommand
A larger list of PEPs that might be hit by an iget:
audit_pep_network_read_header_pre audit_pep_network_read_header_post audit_pep_network_read_body_pre audit_pep_network_read_body_post audit_pep_database_open_pre audit_pep_database_open_post audit_pep_exec_rule_pre audit_pep_exec_microservice_pre audit_pep_exec_microservice_post audit_pep_exec_microservice_pre audit_pep_database_gen_query_access_control_setup_pre audit_pep_database_gen_query_access_control_setup_post audit_pep_exec_microservice_post audit_pep_exec_rule_post audit_pep_database_gen_query_access_control_setup_pre audit_pep_database_gen_query_access_control_setup_post audit_pep_database_gen_query_pre audit_pep_database_get_rcs_pre audit_pep_database_get_rcs_post audit_pep_database_gen_query_post audit_pep_database_gen_query_access_control_setup_pre audit_pep_database_gen_query_access_control_setup_post audit_pep_database_gen_query_pre audit_pep_database_get_rcs_pre audit_pep_database_get_rcs_post audit_pep_database_gen_query_post audit_pep_exec_rule_pre audit_pep_exec_microservice_pre audit_pep_exec_microservice_post audit_pep_exec_rule_post audit_pep_exec_rule_pre audit_pep_exec_microservice_pre audit_pep_exec_microservice_post audit_pep_exec_rule_post audit_pep_exec_rule_pre audit_pep_exec_microservice_pre audit_pep_exec_microservice_post audit_pep_exec_rule_post audit_pep_network_write_body_pre audit_pep_network_write_header_pre audit_pep_network_write_header_post audit_pep_network_write_body_post audit_pep_network_agent_start_pre audit_pep_network_agent_start_post audit_pep_auth_agent_start_pre audit_pep_auth_agent_start_post audit_pep_network_read_header_pre audit_pep_network_read_header_post audit_pep_network_read_body_pre audit_pep_network_read_body_post
audit_pep_auth_request_pre audit_pep_auth_agent_auth_request_pre audit_pep_auth_agent_auth_request_post audit_pep_auth_request_post audit_pep_network_write_body_pre audit_pep_network_write_header_pre audit_pep_network_write_header_post audit_pep_network_write_body_post audit_pep_auth_agent_start_pre audit_pep_auth_agent_start_post audit_pep_network_read_header_pre audit_pep_network_read_header_post audit_pep_network_read_body_pre audit_pep_network_read_body_post audit_pep_auth_response_pre audit_pep_auth_agent_auth_response_pre audit_pep_database_check_auth_pre audit_pep_database_check_auth_post audit_pep_auth_agent_auth_response_post audit_pep_auth_response_post audit_pep_network_write_body_pre audit_pep_network_write_header_pre audit_pep_network_write_header_post audit_pep_network_write_body_post audit_pep_auth_agent_start_pre audit_pep_auth_agent_start_post audit_pep_network_read_header_pre audit_pep_network_read_header_post audit_pep_network_read_body_pre audit_pep_network_read_body_post audit_pep_obj_stat_pre audit_pep_database_gen_query_access_control_setup_pre audit_pep_database_gen_query_access_control_setup_post audit_pep_database_gen_query_pre audit_pep_database_get_rcs_pre audit_pep_database_get_rcs_post audit_pep_database_gen_query_post audit_pep_database_gen_query_access_control_setup_pre audit_pep_database_gen_query_access_control_setup_post audit_pep_database_gen_query_pre audit_pep_database_get_rcs_pre audit_pep_database_get_rcs_post audit_pep_database_gen_query_post audit_pep_obj_stat_post audit_pep_network_write_body_pre audit_pep_network_write_header_pre audit_pep_network_write_header_post audit_pep_network_write_body_post audit_pep_auth_agent_start_pre audit_pep_auth_agent_start_post
audit_pep_network_read_header_pre audit_pep_network_read_header_post audit_pep_network_read_body_pre audit_pep_network_read_body_post audit_pep_data_obj_get_pre audit_pep_database_gen_query_access_control_setup_pre audit_pep_database_gen_query_access_control_setup_post audit_pep_database_gen_query_pre audit_pep_database_get_rcs_pre audit_pep_database_get_rcs_post audit_pep_database_gen_query_post audit_pep_database_gen_query_access_control_setup_pre audit_pep_database_gen_query_access_control_setup_post audit_pep_database_gen_query_pre audit_pep_database_get_rcs_pre audit_pep_database_get_rcs_post audit_pep_database_gen_query_post audit_pep_resource_resolve_hierarchy_pre audit_pep_resource_resolve_hierarchy_post audit_pep_exec_rule_pre audit_pep_exec_microservice_pre audit_pep_exec_microservice_post audit_pep_exec_rule_post audit_pep_resource_open_pre audit_pep_resource_open_post audit_pep_resource_read_pre audit_pep_resource_read_post audit_pep_resource_close_pre audit_pep_resource_close_post audit_pep_exec_rule_pre audit_pep_exec_microservice_pre audit_pep_exec_microservice_post audit_pep_exec_rule_post audit_pep_data_obj_get_post audit_pep_network_write_body_pre audit_pep_network_write_header_pre audit_pep_network_write_header_post audit_pep_network_write_body_post audit_pep_auth_agent_start_pre audit_pep_auth_agent_start_post audit_pep_network_read_header_pre audit_pep_network_read_header_post audit_pep_network_read_body_pre audit_pep_network_read_body_post audit_pep_network_agent_stop_pre audit_pep_network_agent_stop_post audit_pep_database_close_pre audit_pep_database_close_post
connection setup
auth
exists check
data transfer
Example Configuration Stanza (Current)
As is standard for rule engine plugins, the audit plugin is enabled and configured in the rule_engines array in the server configuration file.
"rule_engines": [
{
"instance_name": "irods_rule_engine_plugin-audit_amqp-instance",
"plugin_name": "irods_rule_engine_plugin-audit_amqp",
"plugin_specific_configuration": {
"amqp_location": "ANONYMOUS@localhost:5672",
"amqp_topic": "audit_messages",
"pep_regex_to_match": "pep_(api|resource)_.*"
}
},
{
"instance_name": "irods_rule_engine_plugin-irods_rule_language-instance",
...
"shared_memory_instance": "irods_rule_language_rule_engine"
},
...
Example Configuration Stanza (Current)
"plugin_specific_configuration": {
"amqp_location": "ANONYMOUS@localhost:5672",
"amqp_topic": "audit_messages",
"pep_regex_to_match": "pep_(api|resource)_.*"
}
| Configuration Option | Description |
|---|---|
amqp_location | Endpoint with which to establish AMQP 1.0 connection |
amqp_topic | Path on AMQP endpoint to which to send messages |
pep_regex_to_match | Filter defining which PEPs to generate messages for (EMCAScript regular expression) |
Example Configuration Stanza (NEW)
"plugin_specific_configuration": {
"failsafe_mode": "BLOCK_OPERATION",
"pep_regex_to_match": "pep_(api|resource)_.*",
"amqp_endpoints": [
{
"scheme": "amqp",
"host": "localhost",
"port": 5672
}
],
"amqp_user": "USERNAME",
"amqp_password": "PASSWORD",
"amqp_path": "queues/audit_messages",
"amqp_sasl": {
"enable": true,
"mechanisms": [
"PLAIN",
"ANONYMOUS"
],
"allow_insecure": true
},
"amqp_connection_open_timeout": 30000,
"amqp_sender": {
"delivery_mode": "AT_LEAST_ONCE",
"target": {
"durability_mode": "UNSETTLED_STATE"
}
},
"amqp_durable_messages": true,
"amqp_message_send_timeout": 30000
}Example Configuration Stanza (NEW)
| Configuration Option | Description |
|---|---|
failsafe_mode |
Whether or not to allow operations to continue in the event of an auditing failure |
pep_regex_to_match |
Filter defining which PEPs to generate messages for (EMCAScript regular expression) |
amqp_endpoints |
Array of AMQP endpoints with which to establish AMQP 1.0 connection |
amqp_user | Username with which to authenticate with the AMQP endpoints |
amqp_password | Password with which to authenticate with the AMQP endpoints |
amqp_path | Path on AMQP endpoints to which to send messages |
We now expose most of the Qpid Proton settings as configuration options prefixed with amqp_. Many of these configuration options have a one-to-one relationship with a Qpid Proton setting.
A more complete description of all configuration options can be found in the plugin's README.
ELK Stack Container for Demonstration Purposes
We provide an ELK stack container for demonstrating and testing the audit plugin. It contains the following following services:
-
RabbitMQ - Message broker that receives the AMQP 1.0 messages
-
Elasticsearch - Queryable database that stores the AMQP messages
-
not-logstash - Reads messages from RabbitMQ and writes them to Elasticsearch
-
Kibana - A data visualization dashboarding tool for ElasticSearch​
The container uses a Python script (not-logstash) as a stand-in for logstash, as logstash uses a different, partially-incompatible version of the AMQP protocol.
More information about this container can be found in its README. A prebuilt container image is hosted on Docker Hub as irods/irods_audit_elk_stack.
AMQP 1.0
iRODS
RabbitMQ
not-logstash
ElasticSearch
Kibana
AMQP 1.0
Example: Querying Writes to a Specific Data Object
curl -XGET 'localhost:9201/irods_audit/_search?pretty' -H 'Content-Type: application/json' -d'
{
"_source": [ "@timestamp", "user_user_name", "obj_path" ],
"sort" : [
{"@timestamp":{"order": "asc"}}
],
"size" :10000,
"query": {
"bool": {
"must": [
{ "match": { "rule_name": "pep_api_data_obj_put_pre" } },
{ "match_phrase": { "obj_path": "tempZone/home/rods/stickers.jpg" } }
]
}
}
}' | jq ".hits.hits[] | ._source"ElasticSearch Query:
{
"@timestamp": "1749189407464",
"obj_path": "/tempZone/home/rods/stickers.jpg",
"user_user_name": "rods"
}
{
"@timestamp": "1749191849588",
"obj_path": "/tempZone/home/rods/stickers.jpg",
"user_user_name": "rods"
}
{
"@timestamp": "1749191859408",
"obj_path": "/tempZone/home/rods/stickers.jpg",
"user_user_name": "rods"
}
{
"@timestamp": "1749191873251",
"obj_path": "/tempZone/home/rods/stickers.jpg",
"user_user_name": "rods"
}
{
"@timestamp": "1749203298988",
"obj_path": "/tempZone/home/rods/stickers.jpg",
"user_user_name": "rods"
}Query Result:
In this example, we can see the user rods wrote to stickers.jpg multiple times.
Visualization with Kibana
Questions?