adamwill / 389-ds-base

Forked from 389-ds-base 4 years ago
Clone

9eba2ce Issue 49850 - ldbm_get_nonleaf_ids() slow for databases with many non-leaf entries

Authored and Committed by mreynolds 4 years ago
    Issue 49850 -  ldbm_get_nonleaf_ids() slow for databases with many non-leaf entries
    
    Bug Description:  The logs from an LDIF import indicated that gathering non-leaf IDs
                      for creating the ancestorid index took an enormous amount of time,
                      over 10hrs.  The root cause is that the parentid index btree ordering
                      is lexical, but the IDList being built up from it is sorted numerically.
                      In the existing code, the IDList is maintained in constantly sorted
                      order by idl_insert().
    
    Fix Description:  ldbm_get_nonleaf_ids() switches to idl_append_extend() instead idl_insert()
                      for building up the IDList and then sorts the result only once, using
                      qsort with idl_sort_cmp, after the entire list has been gathered.
    
                      The improvement on identical hardware is for the operation to take 10
                      seconds rather than 10 hours
    
    Patch Author:  Thomas Lackey <telackey@bozemanpass.com>  Thanks for the great contribution!!!
    
    relates: https://pagure.io/389-ds-base/issue/49850
    
    Reviewed by: mreynolds, tbordaz, and firstyear