adamwill / 389-ds-base

Forked from 389-ds-base 4 years ago
Clone

b785fc2 Ticket #47387 - improve logconv.pl performance with large access logs

Authored and Committed by rmeggins 10 years ago
    Ticket #47387 - improve logconv.pl performance with large access logs
    
    https://fedorahosted.org/389/ticket/47387
    Reviewed by: mreynolds (Thanks!)
    Branch: master
    Fix Description: The primary fix is to use tied hashes and arrays backed by
    DB_File database files.  Hashes use DB_HASH and arrays use DB_RECNO.  These
    fixes cut down the time by a factor of 10 or more, and considerably reduce
    the size of the disk files.  There is still room for optimization as given
    by the Devel::NYTProf package, but this should at least give us acceptable
    performance.
    I also did some perl "linting" - running with use warnings and use strict -
    which revealed a lot of code which was cleaned up.
    Platforms tested: RHEL6 x86_64
    Flag Day: no
    Doc impact: no
    (cherry picked from commit 8e35cc8b418bc46fbeb28f4378d24f9c2cab0353)
    (cherry picked from commit 481f2e3aec2834f342c01919d91eb6cfd80f6ca1)
    
        
file modified
+664 -984