Issue #1722: crash in memory cache - sssd

SSSD / sssd

#1722 crash in memory cache

Closed: Fixed None Opened 11 years ago by jhrozek.

I was running some load testing on my laptop when searching for an unrelated issue and I saw this crash:

Program terminated with signal 11, Segmentation fault.
#0  0x000000000042e296 in sss_mc_rm_rec_from_chain (mcc=0x1581750, rec=0x7fd3aaaf1058, 
    hash=1482184792) at src/responder/nss/nsssrv_mmap_cache.c:139

warning: Source file is more recent than executable.
139     slot = mcc->hash_table[hash];
(gdb) bt
#0  0x000000000042e296 in sss_mc_rm_rec_from_chain (mcc=0x1581750, rec=0x7fd3aaaf1058, 
    hash=1482184792) at src/responder/nss/nsssrv_mmap_cache.c:139
#1  0x000000000042e370 in sss_mc_invalidate_rec (mcc=0x1581750, rec=0x7fd3aaaf1058)
    at src/responder/nss/nsssrv_mmap_cache.c:169
#2  0x000000000042e6ef in sss_mc_find_free_slots (mcc=0x1581750, num_slots=13)
    at src/responder/nss/nsssrv_mmap_cache.c:256
#3  0x000000000042e910 in sss_mc_get_record (mcc=0x1581750, rec_len=387, key=
    0x7fff01e84750) at src/responder/nss/nsssrv_mmap_cache.c:331
#4  0x000000000042f063 in sss_mmap_cache_gr_store (mcc=0x1581750, name=0x7fff01e84750, pw=
    0x7fff01e84740, gid=5471, memnum=40, membuf=0x1c8d8da "rlandy", memsize=329)
    at src/responder/nss/nsssrv_mmap_cache.c:566
#5  0x000000000041667e in fill_grent (packet=0x1c91450, dom=0x1579770, nctx=0x1576980, 
    filter_groups=true, gr_mmap_cache=true, msgs=0x1bba2e0, count=0x7fff01e848b0)
    at src/responder/nss/nsssrv_cmd.c:2185
#6  0x00000000004169ec in nss_cmd_getgr_send_reply (dctx=0x1c88b20, filter=true)
    at src/responder/nss/nsssrv_cmd.c:2236
#7  0x000000000041a23d in nss_cmd_getgrgid (cctx=0x1caa7d0)
    at src/responder/nss/nsssrv_cmd.c:2801
#8  0x0000000000435015 in sss_cmd_execute (cctx=0x1caa7d0, sss_cmds=0x6b1ac0 <nss_cmds>)
    at src/responder/common/responder_cmd.c:153
#9  0x000000000043768f in client_recv (cctx=0x1caa7d0)
    at src/responder/common/responder_common.c:293
#10 0x0000000000437ddd in client_fd_handler (ev=0x156f380, fde=0x1c56b90, flags=1, ptr=
    0x1caa7d0) at src/responder/common/responder_common.c:343
#11 0x000000313bc07552 in epoll_event_loop (tvalp=0x7fff01e84b50, std_ev=0x156f450)
    at ../tevent_standard.c:328
#12 std_event_loop_once (ev=<optimized out>, location=<optimized out>)
    at ../tevent_standard.c:567
#13 0x000000313bc04060 in _tevent_loop_once (ev=ev@entry=0x156f380, 
    location=location@entry=0x4a2c9f "src/util/server.c:601") at ../tevent.c:507
#14 0x000000313bc041eb in tevent_common_loop_wait (ev=0x156f380, location=
    0x4a2c9f "src/util/server.c:601") at ../tevent.c:608
#15 0x00000000004761f3 in server_loop (main_ctx=0x1570500) at src/util/server.c:601
#16 0x000000000040a285 in main (argc=1, argv=0x7fff01e84e88)
    at src/responder/nss/nsssrv.c:563
(gdb) bt full
#0  0x000000000042e296 in sss_mc_rm_rec_from_chain (mcc=0x1581750, rec=0x7fd3aaaf1058, 
    hash=1482184792) at src/responder/nss/nsssrv_mmap_cache.c:139
        prev = 0x0
        cur = 0x0
        slot = 6648164
#1  0x000000000042e370 in sss_mc_invalidate_rec (mcc=0x1581750, rec=0x7fd3aaaf1058)
    at src/responder/nss/nsssrv_mmap_cache.c:169
No locals.
#2  0x000000000042e6ef in sss_mc_find_free_slots (mcc=0x1581750, num_slots=13)
    at src/responder/nss/nsssrv_mmap_cache.c:256
        rec = 0x7fd3aaaf1058
        tot_slots = 50000
        cur = 0
        i = 46318276
        t = 46318276
        used = true
#3  0x000000000042e910 in sss_mc_get_record (mcc=0x1581750, rec_len=387, key=
    0x7fff01e84750) at src/responder/nss/nsssrv_mmap_cache.c:331
        old_rec = 0x0
        rec = 0x0
        old_slots = 0
        num_slots = 13
        base_slot = 0
        i = 347
#4  0x000000000042f063 in sss_mmap_cache_gr_store (mcc=0x1581750, name=0x7fff01e84750, pw=
    0x7fff01e84740, gid=5471, memnum=40, membuf=0x1c8d8da "rlandy", memsize=329)
    at src/responder/nss/nsssrv_mmap_cache.c:566
        rec = 0x7fff01e84b50
        data = 0x7fff01e84e80
        gidkey = {str = 0x7fff01e845d0 "5471", len = 5}
        gidstr = "5471\000\000\000\000\261#G"
        data_len = 339
        rec_len = 387
        pos = 0
        ret = 4

simo commented 11 years ago

The hash value is clearly out of bounds here: 1482184792
This is 0x58585858 hex or the string 'XXXX', which looks like uninitialized memory of some sort.

Do you have a way to reproduce this ?
Or maybe you save the original mmap cache file so I can analyze it ?