pigsty-moloch-plugin
github.com/hillar/pigsty-moloch-plugin
Moloch
gihthub.com/aol/moloch
Moloch is an open source, large scale IPv4 packet capturing (PCAP), indexing and database system.
A simple web interface is provided for PCAP browsing, searching, and exporting.
APIs are exposed that allow PCAP data and JSON-formatted session data to be downloaded directly.
Moloch is not meant to replace IDS engines but instead work along side them to store and index all the network traffic in standard PCAP format, providing fast access.
Moloch is built to be deployed across many systems and can scale to handle multiple gigabits/sec of traffic.
Join the mailing list Moloch-fpc, or find help on freenode IRC #moloch-fpc
Follow @Moloch_fpc for the latest in Owl news.
https://moloch.wow.com
Moloch demo! user: moloch pass: moloch
... Moloch IS not IDS
If running suricata or another IDS add an additional two (2) CPUs per interface, and an additional 5G memory (or more depending on IDS requirements)
... running ids on capture
send events to unified2
unified2
Unified2 is a IDS event file format from which programs such as Barnyard2 parse said events to other known and recognizable formats (MySQL, syslog, etc.).
- Suricata writes only in Unified2 format
-
Sourcefire has announced that the Snort release 2.9.3 will only write in Unified2
$ npm install unified2
A small library for unified2 parsing in node.js.
Pigsty
github.com/threatstack/pigsty
PIGSTY
Pigsty is designed as a replacement for Barnyard2.
Pigsty's output architecture is plugin based.
You must install Pigsty, then install and configure any output plugins you need.
pigsty plugins
- pigsty-mysql - MySQL output for Snort ACID schema
- pigsty-sguil - Sends output to sguild
- pigsty-websocket - Websocket output plugin
- pigsty-mongodb - MongoDB output plugin
- pigsty-syslog - Syslog output plugin
- pigsty-irc - IRC output plugin
- pigsty-rest - Rest output plugin by larsx (Eduardo Urias)
pigsty-moloch-plugin
gets event from unified,
looks up session from moloch and tags it
pigsty spooler & parser takes care of unified2 files
plugin queues & dedups events before lookup from es
install
moloch is running
suricata/snort is running
there are event log files on disk
...
$ npm install pigsty
$ npm install pigsty-moloch-plugin
$ pigsty setup
edit config and done ;-)
$ pigsty setup
<edit /etc/pigsty/pigsty.config.js>
Pigsty config
where are unified2 files ?
logs: {
path: '/var/log/suricata',
mode: 'continuous',
bookmark: '/var/run/pigsty/.bookmark'
},
Pigsty config
where are ref files ?
references: {
reference_file: '/etc/suricata/reference.config',
classification_file: '/etc/suricata/classification.config',
gen_file: '/etc/suricata/rules/gen-msg.map',
sid_file: '/etc/suricata/rules/sid-msg.map'
}
pigsty config
what plugins to run ?
output: {
'moloch-plugin': {
...
}
}
.
where is moloch ini file ?
where is moloch DB module ?
'moloch-plugin': {molochConfigFileLocation: '/data/moloch/etc/config.ini',dbModuleLocation : '/data/moloch/viewer/db.js',iniparserModuleLocation: '/data/moloch/../node-iniparser.js', asyncModuleLocation: '/data/moloch/../async.js', keepaliveagentModuleLocation: '/data/moloch/../keep-alive-agent',printStatstoConsole: true,}
tagged sessions...
pigsty-moloch-plugin
By Hillar Aarelaid
pigsty-moloch-plugin
- 3,787
