#833 Intrusion Detection System

Created 8 years ago by lmacken
Modified 3 months ago by lmacken

A couple of weeks ago I did an initial deployment of an Intrusion Detection
System in our infrastructure. It utilizes the prelude stack,
and is currently powered by auditd and prelude-lml events.
Audit gives us a ridiculous amount of power with regarding to
monitoring everything that happens on a system. Prelude-lml,
out of the box using it's pcre plugin, is able to watch a large
variety of service logs, including many things we are running
(asterisk, mod_security, nagios, cacti, PAM, postfix, sendmail, selinux,
shadowutils, sshd, sudo). Prewikka is the web-based frontend
(https://admin.fedoraproject.org/prewikka).

I created a new 'prelude' puppet module that contains the
configuration for audit, auditsp-plugins, libprelude,
prelude-manager, prewikka, prelude-correlator, and prelude-lml.
Turning a node/servergroup into a sensor entails adding the
following to your class definition: 'include prelude::sensor::audisp'
My initial deployment entailed setting up the prelude-manager
and correlator on a single box, and hooking up a single sensor
(bastion).

So, we're now at the point where we can fine tune our audit rules
before we further deploy this infrastructure.
Some things we want to consider:
- Creating security policies for each servergroup
- Define what files/directories/activities we want to monitor on
which machines
- What events to we want to receieve email/sms for?

{{{
[X] Build EPEL-5 i386 and x86_64 packages
[X] Build F-10 prelude stack
[X] Build F8 audit package (1.7.4)
[X] Test the stack setup locally
[X] Put x86_64 packages in f-i repo
[] Build i386 EL-5 packages and push them into puppet
[X] Create prelude puppet module
[X] Setup an audit prelude sensor
[X] Setup prelude-correlator
[X] Setup prelude-lml to monitor log files
[X] Setup prelude-manager
[X] Setup prewikka
[X] Deployment
[X] Setup prewikka and prelude mysql db on db1
[X] Setup prelude-manager and prelude-correlator on noc1
[X] Setup audit and prelude-lml sensor on bastion
[X] Rebuild against audit-1.7.5, which has an improved dispatcher
[X] Tweak audit rules
[X] Fix displayname of nodes
[X] start with the default stig.rules
[X] Enable tracing and bypass rules
[X] Change failure mode to printk1 from panic 2
[X] Add 'ids' keys to the rules, for easy filtering
[
] Setup email notifications for critical issues
[] Zabbiz, nagios, cacti integration
[
] File alteration monitoring
[_] Hook up prelude-lml to our rsyslog heirarchy on log1
: This is a bit more complicated, since the log heirarchy is
: sorted by month/day/, and there is no simple way to point
: prelude-lml at them.
}}}

So, here's an issue that I hit with prelude-lml in our environment

We've got rsyslogd setup on a central logging server where
everything dumps it's logs to. However, the hierarchy that they are
stored in is not very suitable for easily monitoring
(/var/loghosts/$HOST/$YEAR/$MONTH/$DAY/*.log). Ideally, we want
prelude-lml to watch these logs across all hosts, and our central
logging server seems like the best place to do it, but due to the structure, it is very difficult to do so.

Suggestions welcome.

Sorry for the lack of updates, but progress is being made on this task.

Things that we need to do before deploying to the rest of our systems

  • Get our AVCs down to none at all
  • Update to the new audit daemon once it is released
  • Tweak prewikka filters/thresholds so it is manageable.
  • Setup a cron job to flush the prelude db occasionally of alerts that are older than 5 days.

So, we've been making good progress thus far...

  • SELinux AVCs are down to a tolerable level
  • I've been keeping our selinux-policy, audit, and prelude packages up to date
  • I've been flushing our heartbeat/alert db every now and then by hand. We need this to be a cronjob.

I'm going to sit down next week and setup a couple more prelude sensors, most likely on fedorapeople and fedorahosted.

Can you share your prelude puppet module?
It is exactly what I need. I have setup a few machines by hand with about the same setup you described, and I was about to look into puppet myself. It would be a great help if the puppet prelude module and any supporting files/scripts were available for reference.
Thx,
LCB.

We haven't done anything here since prelude. ;(

So, while we still may want to deploy something, I am going to close this and ask for a new ticket in the event we figure out something we want to deploy and have time to manage it, etc.

Login to comment on this ticket.