#162 Infinite loop / spin inside strcmpi_fast, acl_read_access_allowed_on_attr, server DoS
Closed: wontfix None Opened 9 years ago by rmeggins.

https://bugzilla.redhat.com/show_bug.cgi?id=770728

After an attempt to add the "groupOfCertificates" objectclass, with a single
memberCertificateDescription attribute to a particular user, any attempt to do
a search against this particular entry causes the containing thread to spin
inside strcmpi_fast, which is in turn called from deep within
acl_read_access_allowed_on_attr.

Within a short time, all threads are spinning and the server pegs the machine
at 100%, and stops responding to further requests.

The trigger for each thread spin is an attempt by httpd's mod_authnz_ldap to
log in as this particular user. Each hit to httpd causes one thread to spin.

The stacktrace of the spinning thread looks like this:

(gdb) bt
#0  acllas__handle_group_entry (e=0x16913420, callback_data=0x4d05c090)
    at ldap/servers/slapd/intrinsics.h:96
#1  0x00002b34b2c7a3b1 in send_ldap_search_entry_ext (pb=0x16934c30,
    e=<value optimized out>, ectrls=0x0, attrs=0x4d05c0d0, attrsonly=0,
    send_result=0, nentries=0, urls=0x0) at ldap/servers/slapd/result.c:1253
#2  0x00002b34b2c7aa7c in send_ldap_search_entry (pb=0x2b34b6e55b48,
    e=0x1691691d, ectrls=0x66, attrs=0x69, attrsonly=378628368)
    at ldap/servers/slapd/result.c:812
#3  0x00002b34b2c5fc13 in iterate (pb=0x16934c30, be=0x16631cb0,
    send_result=1, pnentries=0x4d057b24, pagesize=-1, pr_statp=0x4d057b18)
    at ldap/servers/slapd/opshared.c:1316
#4  0x00002b34b2c600e2 in send_results_ext (pb=0x16934c30, send_result=1,
    nentries=0x4d057b24, pagesize=-1, pr_stat=0x4d057b18)
    at ldap/servers/slapd/opshared.c:1537
#5  0x00002b34b2c60d15 in op_shared_search (pb=0x16934c30, send_result=1)
    at ldap/servers/slapd/opshared.c:736
#6  0x00002b34b2c6b64a in search_internal_callback_pb (pb=0x16934c30,
    callback_data=0x4d05c090, prc=<value optimized out>,
    psec=<value optimized out>, prec=0)
    at ldap/servers/slapd/plugin_internal_op.c:761
#7  0x00002b34b6e45f92 in acllas__user_ismember_of_group (aclpb=0x1675de10,
    groupDN=<value optimized out>, clientDN=<value optimized out>,
    cache_status=3, clientCert=0x0) at ldap/servers/plugins/acl/acllas.c:2101
---Type <return> to continue, or q <return> to quit---
#8  0x00002b34b6e4a705 in DS_LASGroupDnEval (errp=<value optimized out>,
    attr_name=<value optimized out>, comparator=CMP_OP_EQ,
    attr_pattern=<value optimized out>, cachable=<value optimized out>,
    LAS_cookie=<value optimized out>, subject=0x16738740, resource=0x0,
    auth_info=0x0, global_auth=0x0) at ldap/servers/plugins/acl/acllas.c:921
#9  0x00002b34b7076f00 in ACLEvalAce (errp=0x0, acleval=0x16733e80,
    ace=0x16862930, cachable=0x4d060e38, autharray=0x0, global_auth=0x0)
    at lib/libaccess/oneeval.cpp:254
#10 0x00002b34b70772c6 in ACL_INTEvalTestRights (errp=0x0, acleval=0x16733e80,
    rights=0x4d060f78, map_generic=0x2b34b7059ac0,
    deny_type=<value optimized out>, deny_response=<value optimized out>,
    acl_tag=0x4d060f80, expr_num=0x4d060f9c, cachable=0x4d060ec0)
    at lib/libaccess/oneeval.cpp:785
#11 0x00002b34b7077786 in ACL_EvalTestRights (errp=0x2b34b6e55b48,
    acleval=0x1691691d, rights=0x66, map_generic=0x69, deny_type=0x16916910,
    deny_response=0x0, acl_tag=0x4d060f80, expr_num=0x4d060f9c)
    at lib/libaccess/oneeval.cpp:995
#12 0x00002b34b6e3c6c2 in acl__TestRights (aclpb=0x1675de10, access=4,
    right=<value optimized out>, map_generic=0x2b34b7059ac0,
    result_reason=0x4d061060) at ldap/servers/plugins/acl/acl.c:3068
#13 0x00002b34b6e3f46c in acl_access_allowed (pb=<value optimized out>,
    e=<value optimized out>, attr=0x16929010 "userPassword", val=0x0, access=4)
    at ldap/servers/plugins/acl/acl.c:590
---Type <return> to continue, or q <return> to quit---
#14 0x00002b34b6e401f1 in acl_read_access_allowed_on_attr (pb=0x2aaaac0062f0,
    e=0x1692e610, attr=0x16929010 "userPassword", val=0x0, access=4)
    at ldap/servers/plugins/acl/acl.c:1297
#15 0x00002b34b6e50777 in acl_access_allowed_main (pb=0x2b34b6e55b48,
    e=0x1692e610, attrs=<value optimized out>, val=0x69, access=4, flags=2,
    errbuf=0x0) at ldap/servers/plugins/acl/aclplugin.c:372
#16 0x00002b34b2c6b088 in plugin_call_acl_plugin (pb=0x2aaaac0062f0,
    e=0x1692e610, attrs=0x4d067280, val=0x0, access=4, flags=2, errbuf=0x0)
    at ldap/servers/slapd/plugin_acl.c:90
#17 0x00002b34b2c78df1 in encode_attr_2 (pb=0x2aaaac0062f0, ber=0x16933310,
    e=0x1692e610, vs=0x1692f898, attrsonly=0,
    attribute_type=0x16929010 "userPassword",
    returned_type=0x16934c0b "userPassword") at ldap/servers/slapd/result.c:737
#18 0x00002b34b2c79333 in send_specific_attrs (e=0x1692e610,
    attrs=<value optimized out>, op=0x2aaaac003220, pb=0x2aaaac0062f0,
    ber=0x16933310, attrsonly=0, ldapversion=3, dontsendattr=0x16926650,
    real_attrs_only=0) at ldap/servers/slapd/result.c:1172
#19 0x00002b34b2c79e8d in send_ldap_search_entry_ext (pb=0x2aaaac0062f0,
    e=<value optimized out>, ectrls=0x0, attrs=0x16934b10, attrsonly=0,
    send_result=0, nentries=0, urls=0x0) at ldap/servers/slapd/result.c:1365
#20 0x00002b34b2c7aa7c in send_ldap_search_entry (pb=0x2b34b6e55b48,
    e=0x1691691d, ectrls=0x66, attrs=0x69, attrsonly=378628368)
    at ldap/servers/slapd/result.c:812
---Type <return> to continue, or q <return> to quit---
#21 0x00002b34b2c5fc13 in iterate (pb=0x2aaaac0062f0, be=0x16631cb0,
    send_result=1, pnentries=0x4d06d6a4, pagesize=-1, pr_statp=0x4d06d698)
    at ldap/servers/slapd/opshared.c:1316
#22 0x00002b34b2c600e2 in send_results_ext (pb=0x2aaaac0062f0, send_result=1,
    nentries=0x4d06d6a4, pagesize=-1, pr_stat=0x4d06d698)
    at ldap/servers/slapd/opshared.c:1537
#23 0x00002b34b2c60d15 in op_shared_search (pb=0x2aaaac0062f0, send_result=1)
    at ldap/servers/slapd/opshared.c:736
#24 0x000000000042450f in do_search (pb=0x2aaaac0062f0)
    at ldap/servers/slapd/search.c:393
#25 0x000000000041352c in connection_threadmain ()
    at ldap/servers/slapd/connection.c:611
#26 0x00000032442284ad in ?? () from /usr/lib64/libnspr4.so
#27 0x0000003240a0673d in start_thread () from /lib64/libpthread.so.0
#28 0x00000032402d44bd in clone () from /lib64/libc.so.6

The bottom frames in detail:

(gdb) bt full
#0  acllas__handle_group_entry (e=0x16913420, callback_data=0x4d05c090)
    at ldap/servers/slapd/intrinsics.h:96
        sval = 0x0
        attrVal = 0x4d057b24
        currAttr = 0x169168b0
        nextAttr = 0x169168b0
        n_dn = 0x0
        attrType = 0x16916910 "memberCertificateDescription"
        n = 1292179832
        i = 378752048
#1  0x00002b34b2c7a3b1 in send_ldap_search_entry_ext (pb=0x16934c30,
    e=<value optimized out>, ectrls=0x0, attrs=0x4d05c0d0, attrsonly=0,
    send_result=0, nentries=0, urls=0x0) at ldap/servers/slapd/result.c:1253
        conn = 0x0
        op = 0x16933900
        ber = <value optimized out>
        i = <value optimized out>
        rc = <value optimized out>
        alluserattrs = <value optimized out>
        noattrs = <value optimized out>
        some_named_attrs = <value optimized out>
        dontsendattr = 0x0
        operation = <value optimized out>
---Type <return> to continue, or q <return> to quit---
        real_attrs_only = <value optimized out>
        ctrlp = 0x0
        ecopy = 0x0
        searchctrlp = 0x0
#2  0x00002b34b2c7aa7c in send_ldap_search_entry (pb=0x2b34b6e55b48,
    e=0x1691691d, ectrls=0x66, attrs=0x69, attrsonly=378628368)
    at ldap/servers/slapd/result.c:812
No locals.
#3  0x00002b34b2c5fc13 in iterate (pb=0x16934c30, be=0x16631cb0,
    send_result=1, pnentries=0x4d057b24, pagesize=-1, pr_statp=0x4d057b18)
    at ldap/servers/slapd/opshared.c:1316
        gerentry = 0x2b34b6e54940
        operation = 0x16933900
        rc = 0
        rval = 1
        attrsonly = 0
        done = 0
        e = 0x16913420
        attrs = 0x4d05c0d0
        pr_stat = 0

Stepping through the spin looks like this:

98              } while ( f && (f == l) );
(gdb)
96                      if ( ((l = (unsigned char)(*(src++))) >= 'A') && (l <=
'Z') )
(gdb)
94                      if ( ((f = (unsigned char)(*(dst++))) >= 'A') && (f <=
'Z') )
(gdb)
95                              f -= ('A' - 'a');
(gdb)
96                      if ( ((l = (unsigned char)(*(src++))) >= 'A') && (l <=
'Z') )
(gdb)
97                              l -= ('A' - 'a');
(gdb)
98              } while ( f && (f == l) );
(gdb)
96                      if ( ((l = (unsigned char)(*(src++))) >= 'A') && (l <=
'Z') )
(gdb)
94                      if ( ((f = (unsigned char)(*(dst++))) >= 'A') && (f <=
'Z') )
(gdb)
95                              f -= ('A' - 'a');
(gdb)
96                      if ( ((l = (unsigned char)(*(src++))) >= 'A') && (l <=
'Z') )
(gdb)
97                              l -= ('A' - 'a');
(gdb)
98              } while ( f && (f == l) );
(gdb)
96                      if ( ((l = (unsigned char)(*(src++))) >= 'A') && (l <=
'Z') )
(gdb)
94                      if ( ((f = (unsigned char)(*(dst++))) >= 'A') && (f <=
'Z') )
(gdb)
95                              f -= ('A' - 'a');
(gdb)
96                      if ( ((l = (unsigned char)(*(src++))) >= 'A') && (l <=
'Z') )
(gdb)
97                              l -= ('A' - 'a');
(gdb)
98              } while ( f && (f == l) );

The above code is a macro, which seems to have a bug in it:

ldap/servers/slapd/intrinsics.h

INLINE_DIRECTIVE static int strcmpi_fast(const char * dst, const char * src)
{
        int f,l;
        do {
                if ( ((f = (unsigned char)(*(dst++))) >= 'A') && (f <= 'Z') )
                        f -= ('A' - 'a');
                if ( ((l = (unsigned char)(*(src++))) >= 'A') && (l <= 'Z') )
                        l -= ('A' - 'a');
        } while ( f && (f == l) );
        return(f - l);
}

0001-Ticket-162-Infinite-loop-spin-inside-strcmpi_fast-ac.patch
0001-Ticket-162-Infinite-loop-spin-inside-strcmpi_fast-ac.patch

To ssh://git.fedorahosted.org/git/389/ds.git
aa28a59..1bbbb3e master -> master
commit changeset:1bbbb3e/389-ds-base
Author: Rich Megginson rmeggins@redhat.com
Date: Thu Jan 5 16:49:10 2012 -0700
Reviewed by: nhosoi (Thanks!)
Branch: master
Fix Description: Cannot use continue - have to go to the end of the loop and
get the next attribute - added a label for nextattr and use goto nextattr
instead of continue.
Platforms tested: RHEL6 x86_64
Flag Day: no
Doc impact: no

Added initial screened field value.

Metadata Update from @nkinder:
- Issue assigned to rmeggins
- Issue set to the milestone: 1.2.10

4 years ago

389-ds-base is moving from Pagure to Github. This means that new issues and pull requests
will be accepted only in 389-ds-base's github repository.

This issue has been cloned to Github and is available here:
- https://github.com/389ds/389-ds-base/issues/162

If you want to receive further updates on the issue, please navigate to the github issue
and click on subscribe button.

Thank you for understanding. We apologize for all inconvenience.

Metadata Update from @spichugi:
- Issue close_status updated to: wontfix (was: Fixed)

9 months ago

Login to comment on this ticket.

Metadata