Survey of the project

This project uses a modified unix domain socket output plugin of the network intrusion detection system snort. The alerts generated by snort are read from the unix domain socket by another process called sockserv. This process reads from a socket and sends the alerts via TCP to a central server. On the central server a program called servsock reads these data and writes them via an unix domain socket to a database.

So this project is developed for environments with several remote sensors and one central server gathering the informations. With the normal database output plugin there would be several SELECT and INSERT statements via the network which would slow down the INSERT rate. Additionally in this scheme snort is blocked until all data is spooled to the database. The only reason why you will not loose traffic in between is the buffering in the kernel and/or libpcap. If this results in a buffer overrun in the kernel buffer you will not even notice it. In contrast the libpcap will report the dropped packets if it drops them. (With Linux you should use the libpcap version of Phil Woods at http://public.lanl.gov/cpw/ to get useful statistics.)

The advantage of this method is the complete decoupling of the output processing from snort. The programs sockserv and servsock buffer all alerts in cases of a slow network or a slow database access (or a heavy attack is going on generating a lot of alerts in a very short period). Both programs use two threads, one to receive data and one to forward this data either to the central server or to the database.

In addition to avoid an overrun of the internal buffer of the programs there exists a drop feature. Herein the alerts were dropped before they were forwarded to the central server (sockserv) or the database (servsock). A short description of each dropped alert can be e-mailed to a list of recipients.

Finally there is an alert feature which is able to send alerts as e-mails if the priority reaches a given level. This feature is intended to inform an admin on a high level alert. (There is still a problem in the definition of a high level: Is this a high priority value or a low one? This seemed to be changed sometimes between snort-1.8 and snort-2.0.)

Two further programs/features were added:

• fpg: A false positive generator to test the whole system under a defined load.
• An output option for snort to write statistics to a unix domain socket. This way statistics can be written via a perl script stats.pl to a RRD database.

Documentation

There exist some extended documentation.

There are also some manual pages available (also part of the documentation).

• alert.8
• alert.conf.5
• drop.8
• drop.conf.5
• fpg.8
• getpacket.8
• getpacket.conf.5
• servsock.8
• servsock.conf.5
• sockserv.8

Download

You can download the actual sources here: FLoP-1.6.0

Maybe I will create some binary packages in the future...

Linux (x86) binaries are no longer available. The reason is the usage of the glibc-2.3:

warning: Using 'gethostbyname' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking

The reason seems to be that the glibc still needs the dynamic library to use dlopen. But this is needed at least for host name resolving which can use DNS, files, NIS or something else for name resolution. But these methods are defined on startup and not during compilation.

So maybe I will create statically linked files again...

It will run fine on Linux and Solaris. *BSD (except BSD/OS from WindRiver) should work but I did not find the time to test it in detail. Note: Signals are mainly used for statistics and for a clean exit. Here a clean exit means that all buffered alerts are processed before the program exits.

All other operating systems may work as long as they run a unix system.

Problems will arise on a mixture of 32 bit and 64 bit systems. (One reason is the different size of time_t which is used in the pcap format.)

Problems will arise on a mixture of big and little endian systems. Maybe this will be fixed in the future but actually I have neither the time nor do I have any need for this... Starting with version 1.0.1 any connection from a remote sensor with a different endianess as the central server is now rejected.

Summary

Advantages of FLoP are:

• Decoupling of the output from snort. Snort can work on new packets

   instead of processing the output.

• Buffering of alerts on the sensor. This is useful if you have a shortage on your network to the central server or the servsock process on the central server is not running (maybe it will be restarted due to a change to a newer version...)
• Buffering of alerts on the central server. It is not uncommon that the database (especially MySQL) is hanging during a high input rate or the rate is faster than the database is able to store.
• Fast writing to the database via an unix domain socket.
• E-Mail alerting on high priority alerts.
• Drop feature for the worst case. At least the basic alert informations are still available either via E-Mail or on stdout/syslog.
• Since version 1.0.6 the alerts which should be dropped on the central server if servsock exits are written to a swap file. So this data is still available.
• If alerts have to been dropped because the high water mark was reached then these data are not written to the swap file.

There are also some limitations/disadvantages:

• If you use bpf rules to pre-filter snort traffic then this will not be marked in the database. If you see the alert packet in the database then all you know is the name of the sensor and not the actual configuration. (Otherwise you would get a new sensor entry for each sensor with a changed bpf configuration...)
• Some informations in the database are missing or have to be inserted by another method, for example the classification. This can be easily done with a simple perl script. With version 1.2.2 there is a perl script in the contrib directory called rules.pl which is able to insert these missing parts.
• The traffic between the sensors and the central server is not encrypted. Since our focus is on fast logging we expect that you use a separate network for the communication between the sensors and the central server. (Additionally we assume snort processes running in stealth mode. Therefore a separate network is highly recommended.) Since version 1.5.0 it is possible to use encrypted tunnels like with stunnel or sshd.
• Actually only the databases MySQL and PostgreSQL are supported. I prefer to use the PostgreSQL database which is since version 7.3 not really slower as MySQL but uses less memory
• Maybe you will find some further limitations...