#1722 crash in memory cache
Closed: Fixed None Opened 7 years ago by jhrozek.

I was running some load testing on my laptop when searching for an unrelated issue and I saw this crash:

Program terminated with signal 11, Segmentation fault.
#0  0x000000000042e296 in sss_mc_rm_rec_from_chain (mcc=0x1581750, rec=0x7fd3aaaf1058, 
    hash=1482184792) at src/responder/nss/nsssrv_mmap_cache.c:139

warning: Source file is more recent than executable.
139     slot = mcc->hash_table[hash];
(gdb) bt
#0  0x000000000042e296 in sss_mc_rm_rec_from_chain (mcc=0x1581750, rec=0x7fd3aaaf1058, 
    hash=1482184792) at src/responder/nss/nsssrv_mmap_cache.c:139
#1  0x000000000042e370 in sss_mc_invalidate_rec (mcc=0x1581750, rec=0x7fd3aaaf1058)
    at src/responder/nss/nsssrv_mmap_cache.c:169
#2  0x000000000042e6ef in sss_mc_find_free_slots (mcc=0x1581750, num_slots=13)
    at src/responder/nss/nsssrv_mmap_cache.c:256
#3  0x000000000042e910 in sss_mc_get_record (mcc=0x1581750, rec_len=387, key=
    0x7fff01e84750) at src/responder/nss/nsssrv_mmap_cache.c:331
#4  0x000000000042f063 in sss_mmap_cache_gr_store (mcc=0x1581750, name=0x7fff01e84750, pw=
    0x7fff01e84740, gid=5471, memnum=40, membuf=0x1c8d8da "rlandy", memsize=329)
    at src/responder/nss/nsssrv_mmap_cache.c:566
#5  0x000000000041667e in fill_grent (packet=0x1c91450, dom=0x1579770, nctx=0x1576980, 
    filter_groups=true, gr_mmap_cache=true, msgs=0x1bba2e0, count=0x7fff01e848b0)
    at src/responder/nss/nsssrv_cmd.c:2185
#6  0x00000000004169ec in nss_cmd_getgr_send_reply (dctx=0x1c88b20, filter=true)
    at src/responder/nss/nsssrv_cmd.c:2236
#7  0x000000000041a23d in nss_cmd_getgrgid (cctx=0x1caa7d0)
    at src/responder/nss/nsssrv_cmd.c:2801
#8  0x0000000000435015 in sss_cmd_execute (cctx=0x1caa7d0, sss_cmds=0x6b1ac0 <nss_cmds>)
    at src/responder/common/responder_cmd.c:153
#9  0x000000000043768f in client_recv (cctx=0x1caa7d0)
    at src/responder/common/responder_common.c:293
#10 0x0000000000437ddd in client_fd_handler (ev=0x156f380, fde=0x1c56b90, flags=1, ptr=
    0x1caa7d0) at src/responder/common/responder_common.c:343
#11 0x000000313bc07552 in epoll_event_loop (tvalp=0x7fff01e84b50, std_ev=0x156f450)
    at ../tevent_standard.c:328
#12 std_event_loop_once (ev=<optimized out>, location=<optimized out>)
    at ../tevent_standard.c:567
#13 0x000000313bc04060 in _tevent_loop_once (ev=ev@entry=0x156f380, 
    location=location@entry=0x4a2c9f "src/util/server.c:601") at ../tevent.c:507
#14 0x000000313bc041eb in tevent_common_loop_wait (ev=0x156f380, location=
    0x4a2c9f "src/util/server.c:601") at ../tevent.c:608
#15 0x00000000004761f3 in server_loop (main_ctx=0x1570500) at src/util/server.c:601
#16 0x000000000040a285 in main (argc=1, argv=0x7fff01e84e88)
    at src/responder/nss/nsssrv.c:563
(gdb) bt full
#0  0x000000000042e296 in sss_mc_rm_rec_from_chain (mcc=0x1581750, rec=0x7fd3aaaf1058, 
    hash=1482184792) at src/responder/nss/nsssrv_mmap_cache.c:139
        prev = 0x0
        cur = 0x0
        slot = 6648164
#1  0x000000000042e370 in sss_mc_invalidate_rec (mcc=0x1581750, rec=0x7fd3aaaf1058)
    at src/responder/nss/nsssrv_mmap_cache.c:169
No locals.
#2  0x000000000042e6ef in sss_mc_find_free_slots (mcc=0x1581750, num_slots=13)
    at src/responder/nss/nsssrv_mmap_cache.c:256
        rec = 0x7fd3aaaf1058
        tot_slots = 50000
        cur = 0
        i = 46318276
        t = 46318276
        used = true
#3  0x000000000042e910 in sss_mc_get_record (mcc=0x1581750, rec_len=387, key=
    0x7fff01e84750) at src/responder/nss/nsssrv_mmap_cache.c:331
        old_rec = 0x0
        rec = 0x0
        old_slots = 0
        num_slots = 13
        base_slot = 0
        i = 347
#4  0x000000000042f063 in sss_mmap_cache_gr_store (mcc=0x1581750, name=0x7fff01e84750, pw=
    0x7fff01e84740, gid=5471, memnum=40, membuf=0x1c8d8da "rlandy", memsize=329)
    at src/responder/nss/nsssrv_mmap_cache.c:566
        rec = 0x7fff01e84b50
        data = 0x7fff01e84e80
        gidkey = {str = 0x7fff01e845d0 "5471", len = 5}
        gidstr = "5471\000\000\000\000\261#G"
        data_len = 339
        rec_len = 387
        pos = 0
        ret = 4

The hash value is clearly out of bounds here: 1482184792
This is 0x58585858 hex or the string 'XXXX', which looks like uninitialized memory of some sort.

Do you have a way to reproduce this ?
Or maybe you save the original mmap cache file so I can analyze it ?

Fields changed

owner: somebody => simo
patch: 0 => 1
status: new => assigned

Fields changed

milestone: NEEDS_TRIAGE => SSSD 1.9.4

resolution: => fixed
selected: =>
status: assigned => closed

Metadata Update from @jhrozek:
- Issue assigned to simo
- Issue set to the milestone: SSSD 1.9.4

3 years ago

SSSD is moving from Pagure to Github. This means that new issues and pull requests
will be accepted only in SSSD's github repository.

This issue has been cloned to Github and is available here:
- https://github.com/SSSD/sssd/issues/2764

If you want to receive further updates on the issue, please navigate to the github issue
and click on subscribe button.

Thank you for understanding. We apologize for all inconvenience.

Login to comment on this ticket.

Metadata