Trac Ticket #519 - Search with a complex filter including range search is slow
https://fedorahosted.org/389/ticket/519
Bug description: If a filter contains a range search, the
search retrieves one ID per one idl_fetch and merge it to
the idlist using idl_union, which is slow especially when
the range search result size is large.
Fix description: When the idlist size is larger than nsslapd-
lookthroughlimit, the range search returns ALLID (default
value of nsslapd-lookthroughlimit is 5000). Then, the range
search filter is evaluated before returning to the client.
If the default value of nsslapd-lookthroughlimit can be used,
the search elapsed time is much shorter than generating a
complete idlist in index_range_read_ext. Since the nsslapd-
lookthroughlimit is shared among all the search operations,
larger value might be required for other cases. To have its
own control, this patch introduces a new config parameter
nsslapd-rangelookthroughlimit for the range search.
Also, this patch replaced idl_union in index_range_read_ext
with idl_append_extend and sort the idlist at the end. It
improves the range search, but it is still slower than just
returning ALLID for the large range search.
(cherry picked from commit f026ef0447b18994e0e9b847698703436e874f84)