#9 need directory-based searching
Closed: Fixed None Opened 10 years ago by lcbruzenak.

Some info systems require that the audited data be kept for years.
Now that audit is aggregated, with many clients sending data the logs will fill up fast.
Since rotating hundreds of logs is probably not realistic, they will need to be moved off to a different directory location eventually.
At this point you still want to be able to search through them.
Also - if the audited size is large enough, you need to be able to archive the data in order to free up the space. When needed, it can be restored and that data would need to be searchable.

So, I believe what is needed is to expand the "include rotated logs" option to point to any directory the user specifies.


What about a "chroot" solution?
If we could stuff the audit-viewer and the /var/log/audit/ directory into a chroot-ed area and it would think it was still operating on the real log would that work?
If so, could the whole process be scripted so that the launch would account for the chroot and moving the viewer into place, etc?

Expanding the "include rotated files" option sounds reasonable, although I can't promise any time frame.

When analyzing large data sets, audit-viewer will eventually run into its fundamental limitations: it keeps all audit data in memory, and uses linear search. Large-scale analysis requires using something similar to a database, which probably won't be implemented in audit-viewer.

Re: chroot: You can rebuild audit-viewer to use any log path you like, but I can't see how to make it run-time configurable. It would either be completely insecure (allowing unprivileged users to read any file they want) or completely unusable (requiring system administrator intervention to change the log file directory).

As for the chroot idea, maybe it isn't viable.
Just trying to think of a way to get where I want without code changes.

But even if the audit-viewer could reference an alternate location already policy-protected like the /var/log/audit directory is ... e.g. /var/log/audit-archives - even something that restrictive would help I think.

What concerns me with recompiling is having to duplicate policy, since although I want to be able to access an alternate directory I also want to be able to access the current one as it works now. That makes me think I'd need a separate executable which would need a matching policy pack.?.

I understand the limitations you described; thanks.
I guess something of that scale would be more likely a IPA-type endeavor?

"IPA-type" for larger scale is what I think as well.

Would expanding the "include rotated files" to apply to "files" (as opposed to "system audit log") in the "event source" dialog solve your problem?

I'm not sure I follow; sorry.

What I would want to search would be the following:
1: Currently:
/var/log/audit/ ( This is how it works now ... right?)
2: Nominally:
/var/log/archived-audit/
(this would have some logs I'd have to push into place with a custom trusted program which would go look in my saved month files)
3: Ideally:
/var/log/archived-audit/Jan/2009/
...
/var/log/archived-audit/May/2009/

...
etc.

Ideally, I'd have 1&3 above. I could probably work with 1&2.
Maybe something else too I've not considered.

So if you are saying if you changed the "include rotated files" to "include files in directory:" and then had a directory chooser...that might work since I could make it work like option 1&3 via directory selector.

I was thinking about "specified file + all rotated (using a numeric suffix)". Not "all files in a directory".

I don't think I'll implement handling all files described in 3. (all log files in a directory tree) as a single event source.

The wording here still wasn't clear on my part.

I will be clearer (hopefully) as it looks from the above like a harder task than I was suggesting. I am not thinking about walking a directory tree to grab all the files under a hierarchy.

I am asking if there is a simple way of using the same logic as the "include rotated files" option on an alternate directory.

With the "include rotated files" you search all the logfiles under /var/log/audit/.

What I ask is if, similar to that, it could be applied to a different "system audit log" directory.

This single directory would be comprised of ONLY "audit.log.XX" files.
For example, "/var/log/archived-audit/Jan/2009/" might have:

/var/log/archived-audit/Jan/2009/audit.log.15[[BR]]
/var/log/archived-audit/Jan/2009/audit.log.16[[BR]]
/var/log/archived-audit/Jan/2009/audit.log.17[[BR]]
/var/log/archived-audit/Jan/2009/audit.log.18

I would select "/var/log/archived-audit/Jan/2009" from the (new) "examine alternate log directory" "browse" button.

Is that doable?

Yes, that sounds reasonable.

(To avoid adding a completely separate "directory" option, you'll probably have to select one of the files instead of only the directory. I believe that's a minor detail.)

Support for arbitrary rotated files added in changeset 2384ad6db269. Depends on https://bugzilla.redhat.com/show_bug.cgi?id=504862 (I can work around the bug in audit-viewer, but fixing it in auparse is cleaner.)

Released in audit-viewer 0.5.

Metadata Update from @lcbruzenak:
- Issue assigned to mitr

2 years ago

Login to comment on this ticket.

Metadata