#2531 sssd_be crashes in nested LDAP code with a use-after-free error
Closed: Fixed None Opened 4 years ago by jhrozek.

Ticket was cloned from Red Hat Bugzilla (product Red Hat Enterprise Linux 6): Bug 1173738

Please note that this Bug is private and may not be accessible as it contains confidential Red Hat customer information.

Created attachment 967819
core along with sosreport

Description of problem:

sssd_be crashed at the user log in, if the user is part of nested group.
backtrace of crash:


# bt full
#0  0x0000003e0a032625 in raise (sig=6) at
../nptl/sysdeps/unix/sysv/linux/raise.c:64
        resultvar = 0
        pid = <value optimized out>
        selftid = 21968
#1  0x0000003e0a033e05 in abort () at abort.c:92
        save_stage = 2
        act = {__sigaction_handler = {sa_handler = 0, sa_sigaction = 0},
sa_mask = {__val = {15, 1, 266598359945, 10700928,
              139855995764968, 10476384, 139855995830560, 15, 266598362104,
206158430216, 140735759148224, 140735759148032,
              140735759148240, 140735759148048, 266598384456, 10459856}},
sa_flags = 0, sa_restorer = 0xffffffff}
        sigs = {__val = {32, 0 <repeats 15 times>}}
#2  0x0000003e12802c3c in talloc_abort (reason=0x3e12808348 "Bad talloc magic
value - access after free") at ../talloc.c:317
No locals.
#3  0x0000003e12802dd8 in talloc_abort_access_after_free (ptr=<value optimized
out>) at ../talloc.c:336
No locals.
#4  talloc_chunk_from_ptr (ptr=<value optimized out>) at ../talloc.c:357
        pp = <value optimized out>
        tc = <value optimized out>
#5  talloc_get_name (ptr=<value optimized out>) at ../talloc.c:1153
        tc = 0x0
#6  0x0000003e128057eb in _talloc_get_type_abort (ptr=0x9eb280,
name=0x7f32c2f280e8 "struct tevent_req",
    location=0x7f32c2f38120
"src/providers/ldap/sdap_async_nested_groups.c:935") at ../talloc.c:1206
        pname = <value optimized out>
#7  0x00007f32c2f050b6 in sdap_nested_group_process_done (subreq=0x9fdb60) at
src/providers/ldap/sdap_async_nested_groups.c:935
        state = 0x0
        req = 0x0
        ret = <value optimized out>
        __FUNCTION__ = "sdap_nested_group_process_done"
#8  0x00007f32c2f0289e in sdap_nested_group_single_done (subreq=0x0) at
src/providers/ldap/sdap_async_nested_groups.c:1395
        req = 0x9fdb60
        ret = <value optimized out>
        __FUNCTION__ = "sdap_nested_group_single_done"
#9  0x0000003e10804bde in tevent_req_finish (req=<value optimized out>,
error=<value optimized out>, location=<value optimized out>)
 at ../tevent_req.c:110
No locals.
#10 _tevent_req_error (req=<value optimized out>, error=<value optimized out>,
location=<value optimized out>) at ../tevent_req.c:128
No locals.
#11 0x00007f32c2f05006 in sdap_nested_group_recurse_done (subreq=0x0) at
src/providers/ldap/sdap_async_nested_groups.c:1088
        req = 0x9e8a20
        ret = <value optimized out>
#12 0x0000003e10804bde in tevent_req_finish (req=<value optimized out>,
error=<value optimized out>, location=<value optimized out>)
    at ../tevent_req.c:110
No locals.
#13 _tevent_req_error (req=<value optimized out>, error=<value optimized out>,
location=<value optimized out>) at ../tevent_req.c:128
No locals.
#14 0x00007f32c2f05189 in sdap_nested_group_process_done (subreq=<value
optimized out>)
    at src/providers/ldap/sdap_async_nested_groups.c:975
        state = 0x9e7900
        req = 0x9e7780
        ret = <value optimized out>
        __FUNCTION__ = "sdap_nested_group_process_done"
#15 0x0000003e10804bde in tevent_req_finish (req=<value optimized out>,
error=<value optimized out>, location=<value optimized out>)
    at ../tevent_req.c:110
No locals.
#16 _tevent_req_error (req=<value optimized out>, error=<value optimized out>,
location=<value optimized out>) at ../tevent_req.c:128
No locals.
#17 0x00007f32c2f04ee3 in sdap_nested_group_deref_direct_done (subreq=<value
optimized out>)
    at src/providers/ldap/sdap_async_nested_groups.c:2289
        state = 0xa43020
        req = 0xa45c80
        ret = <value optimized out>
        __FUNCTION__ = "sdap_nested_group_deref_direct_done"
#18 0x0000003e10804bde in tevent_req_finish (req=<value optimized out>,
error=<value optimized out>, location=<value optimized out>)
    at ../tevent_req.c:110
No locals.
#19 _tevent_req_error (req=<value optimized out>, error=<value optimized out>,
location=<value optimized out>) at ../tevent_req.c:128
No locals.
#20 0x00007f32c2ef694b in sdap_deref_search_done (subreq=0x0) at
src/providers/ldap/sdap_async.c:2419
        req = 0xa35d10
        state = 0xa42550
        ret = <value optimized out>
        __FUNCTION__ = "sdap_deref_search_done"
#21 0x0000003e10804bde in tevent_req_finish (req=<value optimized out>,
error=<value optimized out>, location=<value optimized out>)
    at ../tevent_req.c:110
No locals.
#22 _tevent_req_error (req=<value optimized out>, error=<value optimized out>,
location=<value optimized out>) at ../tevent_req.c:128
No locals.
#23 0x0000003e10804bde in tevent_req_finish (req=<value optimized out>,
error=<value optimized out>, location=<value optimized out>)
    at ../tevent_req.c:110
No locals.
#24 _tevent_req_error (req=<value optimized out>, error=<value optimized out>,
location=<value optimized out>) at ../tevent_req.c:128
No locals.
#25 0x00007f32c2ef85a9 in sdap_get_generic_ext_done (op=<value optimized out>,
reply=0x9e8000, error=<value optimized out>,
    pvt=<value optimized out>) at src/providers/ldap/sdap_async.c:1381
        req = 0x9fbb90
        state = 0xa38310
        errmsg = 0x0
        result = <value optimized out>
        ret = 22
        lret = <value optimized out>
        total_count = <value optimized out>
        cookie = {bv_len = 10657392, bv_val = 0x3e12804ca1
"H\205\300I\211\304ttH\205\333A\307D$@p\f\025\350I\307D$H"}
        returned_controls = 0x0
        page_control = <value optimized out>
        __FUNCTION__ = "sdap_get_generic_ext_done"
#26 0x00007f32c2efae2a in sdap_process_message (ev=<value optimized out>,
pvt=<value optimized out>)
    at src/providers/ldap/sdap_async.c:374
msgtype = <value optimized out>
        ret = 0
        reply = 0x9e8000
        op = 0xa384b0
        msgid = 15
#27 sdap_process_result (ev=<value optimized out>, pvt=<value optimized out>)
at src/providers/ldap/sdap_async.c:213
        sh = <value optimized out>
        no_timeout = {tv_sec = 0, tv_usec = 0}
        te = <value optimized out>
        msg = 0xa29e70
        ret = 100
        __FUNCTION__ = "sdap_process_result"
#28 0x0000003e10808ebe in epoll_event_loop (ev=<value optimized out>,
location=<value optimized out>) at ../tevent_epoll.c:736
        fde = <value optimized out>
        flags = <value optimized out>
        mpx_fde = <value optimized out>
        ret = 1
        i = <value optimized out>
        timeout = <value optimized out>
        wait_errno = <value optimized out>
        events = {{events = 1, data = {ptr = 0x9db4f0, fd = 10335472, u32 =
10335472, u64 = 10335472}}}
#29 epoll_event_loop_once (ev=<value optimized out>, location=<value optimized
out>) at ../tevent_epoll.c:931
        epoll_ev = 0x99dd50
        tval = {tv_sec = 5, tv_usec = 999994}
        panic_triggered = false
#30 0x0000003e108072e6 in std_event_loop_once (ev=0x99db40,
location=0x31fea3d5d5 "src/util/server.c:602") at ../tevent_standard.c:112
        glue_ptr = <value optimized out>
        glue = 0x99dc20
        ret = <value optimized out>
#31 0x0000003e1080349d in _tevent_loop_once (ev=0x99db40, location=0x31fea3d5d5
"src/util/server.c:602") at ../tevent.c:530
        ret = <value optimized out>
        nesting_stack_ptr = 0x0
#32 0x0000003e1080351b in tevent_common_loop_wait (ev=0x99db40,
location=0x31fea3d5d5 "src/util/server.c:602") at ../tevent.c:634
        ret = <value optimized out>
#33 0x0000003e10807256 in std_event_loop_wait (ev=0x99db40,
location=0x31fea3d5d5 "src/util/server.c:602") at ../tevent_standard.c:138
        glue_ptr = <value optimized out>
        glue = 0x99dc20
        ret = <value optimized out>
#34 0x00000031fea2b9c3 in server_loop (main_ctx=0x99eeb0) at
src/util/server.c:602
No locals.
#35 0x000000000040a286 in main (argc=4, argv=<value optimized out>) at
src/providers/data_provider_be.c:2856
        opt = <value optimized out>
        pc = <value optimized out>
        be_domain = 0x99b330 "linux.anwb.local"
        srv_name = 0x99b4f0 "sssd[be[linux.anwb.local]]"
        main_ctx = 0x99eeb0
        confdb_path = <value optimized out>
        ret = <value optimized out>
        long_options = {{longName = 0x0, shortName = 0 '\000', argInfo = 4, arg
= 0x624c60, val = 0,
            descrip = 0x41857c "Help options:", argDescrip = 0x0}, {longName =
0x41858a "debug-level", shortName = 100 'd',
            argInfo = 2, arg = 0x624d20, val = 0, descrip = 0x41855b "Debug
level", argDescrip = 0x0}, {
            longName = 0x418596 "debug-to-files", shortName = 102 'f', argInfo
= 0, arg = 0x624c50, val = 0,
            descrip = 0x4199f8 "Send the debug output to files instead of
stderr", argDescrip = 0x0}, {
            longName = 0x4185a5 "debug-timestamps", shortName = 0 '\000',
argInfo = 2, arg = 0x624c40, val = 0,
            descrip = 0x418567 "Add debug timestamps", argDescrip = 0x0},
{longName = 0x4185b6 "debug-microseconds",
            shortName = 0 '\000', argInfo = 2, arg = 0x624d30, val = 0, descrip
= 0x419a30 "Show timestamps with microseconds",
            argDescrip = 0x0}, {longName = 0x41a472 "domain", shortName = 0
'\000', argInfo = 1, arg = 0x7fff98ee6aa8, val = 0,
            descrip = 0x419a58 "Domain of the information provider
(mandatory)", argDescrip = 0x0}, {longName = 0x0,
            shortName = 0 '\000', argInfo = 0, arg = 0x0, val = 0, descrip =
0x0, argDescrip = 0x0}}
        __FUNCTION__ = "main"


Version-Release number of selected component (if applicable):
sssd-common-1.11.6-30.el6

How reproducible:
Customer has not responded/provided this information. However in other
customers environment, incorrectly configured nested group caused this crash.
Fixing nested group mitigated it.

Steps to Reproduce:
1. NA
2.
3.

Actual results:
sssd_be crashes

Expected results:
sssd_be should not crash, user should log in.

Additional info:

I was looking into fedora crashes and ticket https://bugzilla.redhat.com/show_bug.cgi?id=1126557 looks very similar. Unfortunately, reporter is not able to reproduce crash.

blockedby: =>
blocking: =>
changelog: =>
coverity: =>
design: =>
design_review: => 0
feature_milestone: =>
fedora_test_page: =>
mark: no => 0
review: True => 0
selected: =>
testsupdated: => 0

The similar crash occurred with almost recent version in Fedora 20. (sssd-ad-1.11.7-2.fc20)

https://retrace.fedoraproject.org/faf/reports/492920/

Bug is already fixed in sssd-12
In function sdap_nested_group_single_done, tevent_req_error and tevent_req_done was called for the same request.

==29541== Invalid read of size 4
==29541==    at 0x48AC6D7: tevent_req_finish (tevent_req.c:142)
==29541==    by 0x5B15B3A: sdap_nested_group_single_done (sdap_async_nested_groups.c:1395)
==29541==    by 0x48AC6AA: _tevent_req_notify_callback (tevent_req.c:112)
==29541==    by 0x48AC724: tevent_req_finish (tevent_req.c:149)
==29541==    by 0x48AC782: _tevent_req_error (tevent_req.c:167)
==29541==    by 0x5B1956D: sdap_nested_group_recurse_done (sdap_async_nested_groups.c:1088)
==29541==    by 0x48AC6AA: _tevent_req_notify_callback (tevent_req.c:112)
==29541==    by 0x48AC724: tevent_req_finish (tevent_req.c:149)
==29541==    by 0x48AC782: _tevent_req_error (tevent_req.c:167)
==29541==    by 0x5B17605: sdap_nested_group_process_done (sdap_async_nested_groups.c:975)
==29541==    by 0x48AC6AA: _tevent_req_notify_callback (tevent_req.c:112)
==29541==    by 0x48AC724: tevent_req_finish (tevent_req.c:149)
==29541==    by 0x48AC782: _tevent_req_error (tevent_req.c:167)
==29541==    by 0x5B19168: sdap_nested_group_single_step_done (sdap_async_nested_groups.c:1373)
==29541==    by 0x48AC6AA: _tevent_req_notify_callback (tevent_req.c:112)
==29541==    by 0x48AC724: tevent_req_finish (tevent_req.c:149)
==29541==    by 0x48AC782: _tevent_req_error (tevent_req.c:167)
==29541==    by 0x5B17893: sdap_nested_group_lookup_unknown_user_done (sdap_async_nested_groups.c:1921)
==29541==    by 0x48AC6AA: _tevent_req_notify_callback (tevent_req.c:112)
==29541==    by 0x48AC724: tevent_req_finish (tevent_req.c:149)
==29541==    by 0x48AC782: _tevent_req_error (tevent_req.c:167)
==29541==    by 0x5B165CE: sdap_nested_group_lookup_user_done (sdap_async_nested_groups.c:1649)
==29541==    by 0x48AC6AA: _tevent_req_notify_callback (tevent_req.c:112)
==29541==    by 0x48AC724: tevent_req_finish (tevent_req.c:149)
==29541==    by 0x48AC782: _tevent_req_error (tevent_req.c:167)
==29541==    by 0x5B08721: sdap_get_generic_done (sdap_async.c:1603)
==29541==    by 0x48AC6AA: _tevent_req_notify_callback (tevent_req.c:112)
==29541==    by 0x48AC724: tevent_req_finish (tevent_req.c:149)
==29541==    by 0x48AC782: _tevent_req_error (tevent_req.c:167)
==29541==    by 0x5B0AD13: sdap_get_generic_ext_done (sdap_async.c:1431)
==29541==    by 0x5B0A0BE: sdap_process_result (sdap_async.c:374)
==29541==    by 0x48AFC1C: tevent_common_loop_timer_delay (tevent_timed.c:341)
==29541==    by 0x48B0E12: epoll_event_loop_once (tevent_epoll.c:911)
==29541==    by 0x48AF23E: std_event_loop_once (tevent_standard.c:114)
==29541==    by 0x48AB38F: _tevent_loop_once (tevent.c:530)
==29541==    by 0x48AB58B: tevent_common_loop_wait (tevent.c:634)
==29541==    by 0x48AF1BE: std_event_loop_wait (tevent_standard.c:140)
==29541==    by 0x48AB627: _tevent_loop_wait (tevent.c:653)
==29541==    by 0x488A9E8: server_loop (server.c:602)
==29541==    by 0x10CAD6: main (data_provider_be.c:2856)
==29541==  Address 0x60e9ce0 is 128 bytes inside a block of size 445 free'd
==29541==    at 0x4828BCD: free (in /usr/lib/valgrind/vgpreload_memcheck-x86-linux.so)
==29541==    by 0x48B9E8B: _talloc_free (talloc.c:1057)
==29541==    by 0x5B17660: sdap_nested_group_process_done (sdap_async_nested_groups.c:968)
==29541==    by 0x48AC6AA: _tevent_req_notify_callback (tevent_req.c:112)
==29541==    by 0x48AC724: tevent_req_finish (tevent_req.c:149)
==29541==    by 0x48AC782: _tevent_req_error (tevent_req.c:167)
==29541==    by 0x5B15B28: sdap_nested_group_single_done (sdap_async_nested_groups.c:1392)
==29541==    by 0x48AC6AA: _tevent_req_notify_callback (tevent_req.c:112)
==29541==    by 0x48AC724: tevent_req_finish (tevent_req.c:149)
==29541==    by 0x48AC782: _tevent_req_error (tevent_req.c:167)
==29541==    by 0x5B1956D: sdap_nested_group_recurse_done (sdap_async_nested_groups.c:1088)
==29541==    by 0x48AC6AA: _tevent_req_notify_callback (tevent_req.c:112)
==29541==    by 0x48AC724: tevent_req_finish (tevent_req.c:149)
==29541==    by 0x48AC782: _tevent_req_error (tevent_req.c:167)
==29541==    by 0x5B17605: sdap_nested_group_process_done (sdap_async_nested_groups.c:975)
==29541==    by 0x48AC6AA: _tevent_req_notify_callback (tevent_req.c:112)
==29541==    by 0x48AC724: tevent_req_finish (tevent_req.c:149)
==29541==    by 0x48AC782: _tevent_req_error (tevent_req.c:167)
==29541==    by 0x5B19168: sdap_nested_group_single_step_done (sdap_async_nested_groups.c:1373)
==29541==    by 0x48AC6AA: _tevent_req_notify_callback (tevent_req.c:112)
==29541==    by 0x48AC724: tevent_req_finish (tevent_req.c:149)
==29541==    by 0x48AC782: _tevent_req_error (tevent_req.c:167)
==29541==    by 0x5B17893: sdap_nested_group_lookup_unknown_user_done (sdap_async_nested_groups.c:1921)
==29541==    by 0x48AC6AA: _tevent_req_notify_callback (tevent_req.c:112)
==29541==    by 0x48AC724: tevent_req_finish (tevent_req.c:149)
==29541==    by 0x48AC782: _tevent_req_error (tevent_req.c:167)
==29541==    by 0x5B165CE: sdap_nested_group_lookup_user_done (sdap_async_nested_groups.c:1649)
==29541==    by 0x48AC6AA: _tevent_req_notify_callback (tevent_req.c:112)
==29541==    by 0x48AC724: tevent_req_finish (tevent_req.c:149)
==29541==    by 0x48AC782: _tevent_req_error (tevent_req.c:167)
==29541==    by 0x5B08721: sdap_get_generic_done (sdap_async.c:1603)
==29541==    by 0x48AC6AA: _tevent_req_notify_callback (tevent_req.c:112)
==29541==    by 0x48AC724: tevent_req_finish (tevent_req.c:149)
==29541==    by 0x48AC782: _tevent_req_error (tevent_req.c:167)
==29541==    by 0x5B0AD13: sdap_get_generic_ext_done (sdap_async.c:1431)
==29541==    by 0x5B0A0BE: sdap_process_result (sdap_async.c:374)
==29541==    by 0x48AFC1C: tevent_common_loop_timer_delay (tevent_timed.c:341)
==29541==    by 0x48B0E12: epoll_event_loop_once (tevent_epoll.c:911)
==29541==    by 0x48AF23E: std_event_loop_once (tevent_standard.c:114)
==29541==    by 0x48AB38F: _tevent_loop_once (tevent.c:530)
==29541==    by 0x48AB58B: tevent_common_loop_wait (tevent.c:634)
==29541==    by 0x48AF1BE: std_event_loop_wait (tevent_standard.c:140)
==29541==    by 0x48AB627: _tevent_loop_wait (tevent.c:653)
==29541==    by 0x488A9E8: server_loop (server.c:602)
==29541==    by 0x10CAD6: main (data_provider_be.c:2856)

Fields changed

owner: somebody => lslebodn
status: new => assigned

Fields changed

patch: 0 => 1

This ticket only tracks the unit test now.

We also need to backport the fix to sssd-1-11.

Pushed to master:
- be73b26
- 2c7a47b

I also pushed Pavel's original fixes to sssd-1-11:
- f160d64
- 2dd4676

milestone: NEEDS_TRIAGE => SSSD 1.12.4
resolution: => fixed
status: assigned => closed

Metadata Update from @jhrozek:
- Issue assigned to lslebodn
- Issue set to the milestone: SSSD 1.12.4

2 years ago

Login to comment on this ticket.

Metadata