Pigsty – A new Unified2 spooler

At Threat Stack we write a lot of software around NSM, data collection and processing at a large scale to support our cloud based incident response system. We recently decided to write our own custom unified2 spooler and eventually decided to open source it as a Barnyard 2 replacement.

We designed Pigsty (https://github.com/threatstack/pigsty) to be very extensible with a focus of performance. We wanted that have the ability to quickly support new data stores, communication protocols and preprocessing of data post-storage. Pigsty was our answer to that issue and we are happy to announce 0.1.0 is now available to the public under the GPLv3.

Pigsty is very similar to barnyard 2 when it comes to the CLI. Below are the current Pigsty options available.

 Pigsty1

Pigsty2

Pigsty also comes with a configuration file which can be used to store output plugin settings and paths to your reference files.

Pigsty3

Pigsty4

Pigsty5

The Pigsty configuration file is stored as a javascript object and can support multiple output plugin configurations at once. Each output plugin can also accept an array and have multiple sessions open.

Building a new output plugin is really simple.  Please see the example plugin (https://github.com/threatstack/pigsty-example-plugin) hich simple logs the event to console.  We have examples of outputting to MongoDB, SocketIO, and others right on our GitHub page.

Pigsty6

We built an example application using Pigsty and the Mysql/Websocket output plugins. This example application (http://snorby.org:3009/) is connected to the IDS sensor for http://demo.snorby.org and clicking `send attack` in the top right corner will trigger rules to fire for demo purposes.  All events are streamed in realtime.