January 14-16 2020
CINES
Montpellier, France
Jason Coposky
@jason_coposky
Executive Director, iRODS Consortium
Auditing
Auditing
Tracking What We've Done
Remember at the end of the "Getting Started" section:
Tracking What We've Done
Tracking What We've Done
You have a visualization of what has happened in your iRODS zone for the day.
You can see the bytes written and received, connections, top users, etc.
These are just a sample of what can be visualized.
All of the data is in the Elastic database and can be queried for additional interesting patterns or characteristics.
Tracking Origin of a File
Now let's say we want to track the origin (provenance) of some files in our system.
We have PEPs stored in our Elastic database that provide an audit trail for us.
Before we get started, let's install jq so that we can parse the JSON output of an elasticsearch query.
sudo apt-get -y install jq
Tracking Who Wrote to the File
curl -XGET 'localhost:9200/irods_audit/_search?pretty' -H 'Content-Type: application/json' -d'
{
"_source": [ "@timestamp", "user_user_name", "obj_path" ],
"sort" : [
{"@timestamp":{"order": "asc"}}
],
"size" :10000,
"query": {
"bool": {
"must": [
{ "match": { "rule_name": "audit_pep_api_data_obj_put_pre" } },
{ "match_phrase": { "obj_path": "tempZone/home/rods/stickers.jpg" } }
]
}
}
}' | jq ".hits.hits[] | ._source"
Search for put activity on /tempZone/home/rods/stickers.jpg
Tracking Who Wrote to the File
This query returns the following five records showing the user rods put stickers.jpg five times:
{ "@timestamp": "2018-05-30T20:13:01.331Z", "obj_path": "/tempZone/home/rods/stickers.jpg", "user_user_name": "rods" } { "@timestamp": "2018-05-30T21:02:59.350Z", "obj_path": "/tempZone/home/rods/stickers.jpg", "user_user_name": "rods" } { "@timestamp": "2018-05-30T21:03:16.370Z", "obj_path": "/tempZone/home/rods/stickers.jpg", "user_user_name": "rods" } { "@timestamp": "2018-05-30T21:03:31.671Z", "obj_path": "/tempZone/home/rods/stickers.jpg", "user_user_name": "rods" } { "@timestamp": "2018-05-30T21:12:01.143Z", "obj_path": "/tempZone/home/rods/stickers.jpg", "user_user_name": "rods" }
Tracking Read Access to the File
curl -XGET 'localhost:9200/irods_audit/_search?pretty' -H 'Content-Type: application/json' -d'
{
"_source": [ "@timestamp", "user_user_name", "obj_path" ],
"sort" : [
{"@timestamp":{"order": "asc"}}
],
"size" :10000,
"query": {
"bool": {
"must": [
{ "match": { "rule_name": "audit_pep_api_data_obj_get_pre" } },
{ "match_phrase": { "obj_path": "tempZone/home/rods/stickers.jpg" } }
]
}
}
}' | jq ".hits.hits[] | ._source"
There are two reads from user rods:
{ "@timestamp": "2018-05-30T21:05:54.588Z", "obj_path": "/tempZone/home/rods/stickers.jpg", "user_user_name": "rods" } { "@timestamp": "2018-05-30T21:07:08.202Z", "obj_path": "/tempZone/home/rods/stickers.jpg", "user_user_name": "rods" }
Search for read activity on /tempZone/home/rods/stickers.jpg
Look for all the "pre" PEPs
Search for all the "pre" PEPs that have been executed today, but exclude any authentication PEPs
curl -XGET 'localhost:9200/irods_audit/_search?pretty' -H 'Content-Type: application/json' -d' { "_source": [ "@timestamp", "rule_name" ], "sort" : [ {"@timestamp":{"order": "asc"}} ], "size" :10000, "query": { "bool": { "must" : { "regexp": {"rule_name": "audit_pep_api_.*_pre"} }, "must_not" : { "regexp": {"rule_name": "audit_pep_api_auth_.*_pre"} } } } }' | jq ".hits.hits[] | ._source"
Questions?