62e93bc Ticket 1 - pre-normalize filter and pre-compile substring regex - and other optimizations

Authored and Committed by rmeggins 12 years ago
34 files changed. 896 lines added. 204 lines removed.
ldap/servers/plugins/syntaxes/bitstring.c
file modified
+25 -1
ldap/servers/plugins/syntaxes/ces.c
file modified
+26 -1
ldap/servers/plugins/syntaxes/cis.c
file modified
+26 -2
ldap/servers/plugins/syntaxes/deliverymethod.c
file modified
+26 -1
ldap/servers/plugins/syntaxes/dn.c
file modified
+25 -2
ldap/servers/plugins/syntaxes/facsimile.c
file modified
+26 -1
ldap/servers/plugins/syntaxes/guide.c
file modified
+26 -1
ldap/servers/plugins/syntaxes/int.c
file modified
+27 -2
ldap/servers/plugins/syntaxes/nameoptuid.c
file modified
+25 -1
ldap/servers/plugins/syntaxes/numericstring.c
file modified
+26 -1
ldap/servers/plugins/syntaxes/sicis.c
file modified
+26 -1
ldap/servers/plugins/syntaxes/string.c
file modified
+161 -104
ldap/servers/plugins/syntaxes/syntax.h
file modified
+1 -0
ldap/servers/plugins/syntaxes/tel.c
file modified
+26 -1
ldap/servers/plugins/syntaxes/teletex.c
file modified
+26 -1
ldap/servers/plugins/syntaxes/telex.c
file modified
+26 -2
ldap/servers/plugins/syntaxes/value.c
file modified
+4 -0
ldap/servers/slapd/attr.c
file modified
+4 -0
ldap/servers/slapd/ava.c
file modified
+2 -0
ldap/servers/slapd/back-ldbm/back-ldbm.h
file modified
+1 -0
ldap/servers/slapd/back-ldbm/filterindex.c
file modified
+1 -0
ldap/servers/slapd/back-ldbm/ldbm_attr.c
file modified
+2 -39
ldap/servers/slapd/back-ldbm/ldbm_search.c
file modified
+135 -0
ldap/servers/slapd/filter.c
file modified
+98 -34
ldap/servers/slapd/filtercmp.c
file modified
+1 -0
ldap/servers/slapd/filterentry.c
file modified
+8 -5
ldap/servers/slapd/pblock.c
file modified
+28 -0
ldap/servers/slapd/plugin_syntax.c
file modified
+40 -0
ldap/servers/slapd/proto-slap.h
file modified
+2 -2
ldap/servers/slapd/slap.h
file modified
+5 -0
ldap/servers/slapd/slapi-plugin.h
file modified
+36 -0
ldap/servers/slapd/slapi-private.h
file modified
+2 -0
ldap/servers/slapd/utf8compare.c
file modified
+2 -2
ldap/servers/slapd/value.c
file modified
+1 -0
    Ticket 1 - pre-normalize filter and pre-compile substring regex - and other optimizations
    
    When processing large search filters which are applied to every entry in
    the search result set, the filter is normalized anew each time a new entry
    is tested.  For substring filters, a regular expression must be created,
    compiled, and freed each time the substring filter is tested, in addition
    to normalizing the values.  For example, if the search filter contains
    1000 substring sub-filters, for each entry tested with the filter, this
    will require 1000 filter normalizations followed by 1000 regex creation,
    compilation, and cleanup.  If there are 1000 entries in the search result
    set, this will require a million such operations.
    
    The solution is to "pre-compile" the search filter - perform all necessary
    normalizations and compiling of the regular expressions used in the
    filter once we know the search will go through.
    
    struct subfilt and struct ava have "private" members which weren't being
    used for anything.  For subfilt, the private field is used to store the
    pre-compiled regex to pass to the syntax filter code.  For ava, the
    private field is used to store the flags to specify if the type and/or
    value is normalized.
    
    Try to avoid normalization wherever possible.  slapi_value has a v_flags
    field which is used to specify if the value is normalized.  Check this
    before we attempt to normalize a value.  If we are creating a new
    slapi_value, set the normalized flag if the new value is already
    normalized.  Have to make sure that Slapi_Value structures are always
    initialized correctly.
    
    When examining the filter string, do not convert it to lower case first -
    just use strcasestr - note that even though the string may be utf8,
    strcasestr will still work, because we are searching for ascii characters.
    Use PL_strcasestr because the system strcasestr causes valgrind to
    print uninitialized memory errors.
    
    Eliminate some uses of sprintf where a simple char assignment will suffice.
    Reviewed by: nhosoi (Thanks!)
    
        
file modified
+4 -0
file modified
+2 -0
file modified
+98 -34
file modified
+28 -0
file modified
+5 -0
file modified
+1 -0