June 25-28, 2019
iRODS User Group Meeting 2019
Utrecht, Netherlands
Justin James
Applications Engineer
iRODS Consortium
Administration
Auditing
Administration
Auditing
Tracking What We've Done
Remember at the end of the "Getting Started" section:
- We installed an audit plugin that generated events for every dynamic policy enforcement point executed
- We started a docker container with the following software:
- RabbitMQ - message broker to catch events
- Logstash - to read the events and write to DB
- Elasticsearch - database to store the events
- Kibana - visualization tool
Tracking What We've Done
- Visit http://<ip> where ip is the public IP for your VM
- Click on Dashboard
- Select "Today" for the time period at the top of the screen
Tracking What We've Done
You have a visualization of what has happened in your iRODS zone for the day.
You can see the bytes written and received, connections, top users, etc.
These are just a sample of what can be visualized.
All of the data is in the Elastic database and can be queried for additional interesting patterns or characteristics.
Tracking Origin of a File
Now let's say we want to track the origin (provenance) of some files in our system.
We have PEPs stored in our Elastic database that provide an audit trail for us.
Before we get started, let's install jq so that we can parse the JSON output of an elasticsearch query.
sudo apt-get -y install jq
Tracking Who Wrote to the File
curl -XGET 'localhost:9200/irods_audit/_search?pretty' -H 'Content-Type: application/json' -d'
{
"_source": [ "@timestamp", "user_user_name", "obj_path" ],
"sort" : [
{"@timestamp":{"order": "asc"}}
],
"size" :10000,
"query": {
"bool": {
"must": [
{ "match": { "rule_name": "audit_pep_api_data_obj_put_pre" } },
{ "match_phrase": { "obj_path": "tempZone/home/rods/stickers.jpg" } }
]
}
}
}' | jq ".hits.hits[] | ._source"
Search for put activity on /tempZone/home/rods/stickers.jpg
Tracking Who Wrote to the File
This query returns the following five records showing the user rods put stickers.jpg five times:
{ "@timestamp": "2018-05-30T20:13:01.331Z", "obj_path": "/tempZone/home/rods/stickers.jpg", "user_user_name": "rods" } { "@timestamp": "2018-05-30T21:02:59.350Z", "obj_path": "/tempZone/home/rods/stickers.jpg", "user_user_name": "rods" } { "@timestamp": "2018-05-30T21:03:16.370Z", "obj_path": "/tempZone/home/rods/stickers.jpg", "user_user_name": "rods" } { "@timestamp": "2018-05-30T21:03:31.671Z", "obj_path": "/tempZone/home/rods/stickers.jpg", "user_user_name": "rods" } { "@timestamp": "2018-05-30T21:12:01.143Z", "obj_path": "/tempZone/home/rods/stickers.jpg", "user_user_name": "rods" }
Tracking Read Access to the File
curl -XGET 'localhost:9200/irods_audit/_search?pretty' -H 'Content-Type: application/json' -d'
{
"_source": [ "@timestamp", "user_user_name", "obj_path" ],
"sort" : [
{"@timestamp":{"order": "asc"}}
],
"size" :10000,
"query": {
"bool": {
"must": [
{ "match": { "rule_name": "audit_pep_api_data_obj_get_pre" } },
{ "match_phrase": { "obj_path": "tempZone/home/rods/stickers.jpg" } }
]
}
}
}' | jq ".hits.hits[] | ._source"
There are two reads from user rods:
{ "@timestamp": "2018-05-30T21:05:54.588Z", "obj_path": "/tempZone/home/rods/stickers.jpg", "user_user_name": "rods" } { "@timestamp": "2018-05-30T21:07:08.202Z", "obj_path": "/tempZone/home/rods/stickers.jpg", "user_user_name": "rods" }
Search for read activity on /tempZone/home/rods/stickers.jpg
Look for all the "pre" PEPs
Search for all the "pre" PEPs that have been executed today, but exclude any authentication PEPs
curl -XGET 'localhost:9200/irods_audit/_search?pretty' -H 'Content-Type: application/json' -d' { "_source": [ "@timestamp", "rule_name" ], "sort" : [ {"@timestamp":{"order": "asc"}} ], "size" :10000, "query": { "bool": { "must" : { "regexp": {"rule_name": "audit_pep_api_.*_pre"} }, "must_not" : { "regexp": {"rule_name": "audit_pep_api_auth_.*_pre"} } } } }' | jq ".hits.hits[] | ._source"
UGM 2019 - Administration Auditing
By iRODS Consortium
UGM 2019 - Administration Auditing
iRODS User Group Meeting 2019 - Administration Training Module
- 1,635