f6980a4 Ticket 519 - Search with a complex filter including range search is slow

Authored and Committed by mreynolds 11 years ago
    Ticket 519 - Search with a complex filter including range search is slow
    
    https://fedorahosted.org/389/ticket/519
    
    Bug description: If a filter contains a range search, the
    search retrieves one ID per one idl_fetch and merge it to
    the idlist using idl_union, which is slow especially when
    the range search result size is large.
    
    Fix description: When the idlist size is larger than nsslapd-
    lookthroughlimit, the range search returns ALLID (default
    value of nsslapd-lookthroughlimit is 5000).  Then, the range
    search filter is evaluated before returning to the client.
    If the default value of nsslapd-lookthroughlimit can be used,
    the search elapsed time is much shorter than generating a
    complete idlist in index_range_read_ext.  Since the nsslapd-
    lookthroughlimit is shared among all the search operations,
    larger value might be required for other cases.  To have its
    own control, this patch introduces a new config parameter
    nsslapd-rangelookthroughlimit for the range search.
    
    Also, this patch replaced idl_union in index_range_read_ext
    with idl_append_extend and sort the idlist at the end.  It
    improves the range search, but it is still slower than just
    returning ALLID for the large range search.