January 14-16 2020
CINES
Montpellier, France
Jason Coposky
@jason_coposky
Executive Director, iRODS Consortium
Auditing
Auditing
Tracking What We've Done
Remember at the end of the "Getting Started" section:
- We installed an audit plugin that generated events for every dynamic policy enforcement point executed
- We started a docker container with the following software:
- RabbitMQ - message broker to catch events
- Logstash - to read the events and write to DB
- Elasticsearch - database to store the events
- Kibana - visualization tool
Tracking What We've Done
- Visit http://<ip> where ip is the public IP for your VM
- Click on Dashboard
- Select "Today" for the time period at the top of the screen
Tracking What We've Done
You have a visualization of what has happened in your iRODS zone for the day.
You can see the bytes written and received, connections, top users, etc.
These are just a sample of what can be visualized.
All of the data is in the Elastic database and can be queried for additional interesting patterns or characteristics.
Tracking Origin of a File
Now let's say we want to track the origin (provenance) of some files in our system.
We have PEPs stored in our Elastic database that provide an audit trail for us.
Before we get started, let's install jq so that we can parse the JSON output of an elasticsearch query.
sudo apt-get -y install jq
Tracking Who Wrote to the File
curl -XGET 'localhost:9200/irods_audit/_search?pretty' -H 'Content-Type: application/json' -d'
{
"_source": [ "@timestamp", "user_user_name", "obj_path" ],
"sort" : [
{"@timestamp":{"order": "asc"}}
],
"size" :10000,
"query": {
"bool": {
"must": [
{ "match": { "rule_name": "audit_pep_api_data_obj_put_pre" } },
{ "match_phrase": { "obj_path": "tempZone/home/rods/stickers.jpg" } }
]
}
}
}' | jq ".hits.hits[] | ._source"
Search for put activity on /tempZone/home/rods/stickers.jpg
Tracking Who Wrote to the File
This query returns the following five records showing the user rods put stickers.jpg five times:
{ "@timestamp": "2018-05-30T20:13:01.331Z", "obj_path": "/tempZone/home/rods/stickers.jpg", "user_user_name": "rods" } { "@timestamp": "2018-05-30T21:02:59.350Z", "obj_path": "/tempZone/home/rods/stickers.jpg", "user_user_name": "rods" } { "@timestamp": "2018-05-30T21:03:16.370Z", "obj_path": "/tempZone/home/rods/stickers.jpg", "user_user_name": "rods" } { "@timestamp": "2018-05-30T21:03:31.671Z", "obj_path": "/tempZone/home/rods/stickers.jpg", "user_user_name": "rods" } { "@timestamp": "2018-05-30T21:12:01.143Z", "obj_path": "/tempZone/home/rods/stickers.jpg", "user_user_name": "rods" }
Tracking Read Access to the File
curl -XGET 'localhost:9200/irods_audit/_search?pretty' -H 'Content-Type: application/json' -d'
{
"_source": [ "@timestamp", "user_user_name", "obj_path" ],
"sort" : [
{"@timestamp":{"order": "asc"}}
],
"size" :10000,
"query": {
"bool": {
"must": [
{ "match": { "rule_name": "audit_pep_api_data_obj_get_pre" } },
{ "match_phrase": { "obj_path": "tempZone/home/rods/stickers.jpg" } }
]
}
}
}' | jq ".hits.hits[] | ._source"
There are two reads from user rods:
{ "@timestamp": "2018-05-30T21:05:54.588Z", "obj_path": "/tempZone/home/rods/stickers.jpg", "user_user_name": "rods" } { "@timestamp": "2018-05-30T21:07:08.202Z", "obj_path": "/tempZone/home/rods/stickers.jpg", "user_user_name": "rods" }
Search for read activity on /tempZone/home/rods/stickers.jpg
Look for all the "pre" PEPs
Search for all the "pre" PEPs that have been executed today, but exclude any authentication PEPs
curl -XGET 'localhost:9200/irods_audit/_search?pretty' -H 'Content-Type: application/json' -d' { "_source": [ "@timestamp", "rule_name" ], "sort" : [ {"@timestamp":{"order": "asc"}} ], "size" :10000, "query": { "bool": { "must" : { "regexp": {"rule_name": "audit_pep_api_.*_pre"} }, "must_not" : { "regexp": {"rule_name": "audit_pep_api_auth_.*_pre"} } } } }' | jq ".hits.hits[] | ._source"
Questions?
CINES 2020 - Auditing
By jason coposky
CINES 2020 - Auditing
CINES 2020 Training Module
- 1,199