0a54aa6 Trac Ticket #519 - Search with a complex filter including range search is slow

Authored and Committed by nhosoi 11 years ago
    Trac Ticket #519 - Search with a complex filter including range search is slow
    
    https://fedorahosted.org/389/ticket/519
    
    Bug description: If a filter contains a range search, the
    search retrieves one ID per one idl_fetch and merge it to
    the idlist using idl_union, which is slow especially when
    the range search result size is large.
    
    Fix description: When the idlist size is larger than nsslapd-
    lookthroughlimit, the range search returns ALLID (default
    value of nsslapd-lookthroughlimit is 5000).  Then, the range
    search filter is evaluated before returning to the client.
    If the default value of nsslapd-lookthroughlimit can be used,
    the search elapsed time is much shorter than generating a
    complete idlist in index_range_read_ext.  Since the nsslapd-
    lookthroughlimit is shared among all the search operations,
    larger value might be required for other cases.  To have its
    own control, this patch introduces a new config parameter
    nsslapd-rangelookthroughlimit for the range search.
    
    Also, this patch replaced idl_union in index_range_read_ext
    with idl_append_extend and sort the idlist at the end.  It
    improves the range search, but it is still slower than just
    returning ALLID for the large range search.
    (cherry picked from commit f026ef0447b18994e0e9b847698703436e874f84)