Description of problem: Crash is observed in automated tests in indexes test suite (test case indexes3). How reproducible: always Additional info: core_backtrace: :{ "signal": 11 :, "executable": "/usr/sbin/ns-slapd" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140696621491566 : , "build_id": "32976b8813b497dac812200e859d7455f9058c61" : , "build_id_offset": 623982 : , "function_name": "objset_first_obj" : , "file_name": "/usr/lib64/dirsrv/libslapd.so.0.0.0" : } : , { "address": 140696424276440 : , "build_id": "b3e230bd8e99fd9c94e6287dd6528ae60312f0fd" : , "build_id_offset": 143832 : , "function_name": "dblayer_terminate" : , "file_name": "/usr/lib64/dirsrv/plugins/libback-ldbm.so" : } : , { "address": 140696424263570 : , "build_id": "b3e230bd8e99fd9c94e6287dd6528ae60312f0fd" : , "build_id_offset": 130962 : , "function_name": "ldbm_back_cleanup" : , "file_name": "/usr/lib64/dirsrv/plugins/libback-ldbm.so" : } : , { "address": 140696621209691 : , "build_id": "32976b8813b497dac812200e859d7455f9058c61" : , "build_id_offset": 342107 : , "function_name": "be_cleanupall" : , "file_name": "/usr/lib64/dirsrv/libslapd.so.0.0.0" : } : , { "address": 140696626657880 : , "build_id": "0e11d0e385de2114fcd2fc170b8e6c6000d9977a" : , "build_id_offset": 133720 : , "function_name": "slapd_daemon" : , "file_name": "/usr/sbin/ns-slapd" : } : , { "address": 140696626598147 : , "build_id": "0e11d0e385de2114fcd2fc170b8e6c6000d9977a" : , "build_id_offset": 73987 : , "function_name": "main" : , "file_name": "/usr/sbin/ns-slapd" : } ] : } ] :}
A possible test case is to create two suffixes. Then both backends will refer the same be_database (see bz). Then at shutdown plugin_free will be called for each backends that will do a double free.
This is likely related to the change https://fedorahosted.org/389/ticket/48872, that call plugin_free for each backends.
attachment ticket48891_test.py
attachment 0001-Ticket-48891-ns-slapd-crashes-during-the-shutdown-af.patch
I'm happy for this to be reverted, I will keep this test in mind when I come back to this. The intent was to prevent a memory leak on shutdown.
Ack from me.
git '''merge''' ticket_48891 Updating 250a49d..7b8fc84 Fast-forward dirsrvtests/tests/tickets/ticket48891_test.py | 175 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ ldap/servers/slapd/backend_manager.c | 1 - ldap/servers/slapd/daemon.c | 2 +- ldap/servers/slapd/plugin.c | 7 +--- ldap/servers/slapd/proto-slap.h | 1 - 5 files changed, 178 insertions(+), 8 deletions(-) create mode 100644 dirsrvtests/tests/tickets/ticket48891_test.py
git '''push''' origin master Counting objects: 13, done. Delta compression using up to 8 threads. Compressing objects: 100% (12/12), done. Writing objects: 100% (13/13), 3.29 KiB | 0 bytes/s, done. Total 13 (delta 10), reused 1 (delta 1) To ssh://git.fedorahosted.org/git/389/ds.git 250a49d..7b8fc84 master -> master
commit 7b8fc84 Author: Thierry Bordaz tbordaz@redhat.com Date: Tue Jun 21 10:28:03 2016 +0200
The bug fix https://fedorahosted.org/389/ticket/48872 only exists in 1.3.5 branch. Closing the ticket
My build from the master branch shows Invalid read. {{{ ==26679== Invalid read of size 8 ==26679== at 0x4C696CC: be_cleanupall (backend_manager.c:318) ==26679== by 0x128340: slapd_daemon (daemon.c:1345) ==26679== by 0x13198E: main (main.c:1143) ==26679== Address 0x11673ac0 is 304 bytes inside a block of size 696 free'd ==26679== at 0x4A07D6A: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==26679== by 0x4C6B916: slapi_ch_free (ch_malloc.c:292) ==26679== by 0x4CD6739: plugin_free (plugin.c:2732) ==26679== by 0x4CD4C96: plugin_closeall (plugin.c:1959) ==26679== by 0x1282E6: slapd_daemon (daemon.c:1321) ==26679== by 0x13198E: main (main.c:1143) ==26679== ==26679== Invalid read of size 8 ==26679== at 0x4C697A8: be_cleanupall (backend_manager.c:325) ==26679== by 0x128340: slapd_daemon (daemon.c:1345) ==26679== by 0x13198E: main (main.c:1143) ==26679== Address 0x11673ac0 is 304 bytes inside a block of size 696 free'd ==26679== at 0x4A07D6A: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==26679== by 0x4C6B916: slapi_ch_free (ch_malloc.c:292) ==26679== by 0x4CD6739: plugin_free (plugin.c:2732) ==26679== by 0x4CD4C96: plugin_closeall (plugin.c:1959) ==26679== by 0x1282E6: slapd_daemon (daemon.c:1321) ==26679== by 0x13198E: main (main.c:1143) ==26679== ==26679== Invalid read of size 8 ==26679== at 0x4CCA458: slapi_pblock_get (pblock.c:442) ==26679== by 0x127B377A: ldbm_back_cleanup (cleanup.c:24) ==26679== by 0x4C697BF: be_cleanupall (backend_manager.c:325) ==26679== by 0x128340: slapd_daemon (daemon.c:1345) ==26679== by 0x13198E: main (main.c:1143) ==26679== Address 0x11673990 is 0 bytes inside a block of size 696 free'd ==26679== at 0x4A07D6A: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==26679== by 0x4C6B916: slapi_ch_free (ch_malloc.c:292) ==26679== by 0x4CD6739: plugin_free (plugin.c:2732) ==26679== by 0x4CD4C96: plugin_closeall (plugin.c:1959) ==26679== by 0x1282E6: slapd_daemon (daemon.c:1321) ==26679== by 0x13198E: main (main.c:1143) }}} The patch 0001-Ticket-48891-ns-slapd-crashes-during-the-shutdown-af.patch​ eliminated some changes made by the fix in #48872. But it seems we still have to skip the backend release in plugin_closeall... By resurrecting this code, valgrind stops complaining the invalid read... {{{ diff --git a/ldap/servers/slapd/daemon.c b/ldap/servers/slapd/daemon.c index 862f6e7..90d1f7d 100644 --- a/ldap/servers/slapd/daemon.c +++ b/ldap/servers/slapd/daemon.c @@ -1318,7 +1318,7 @@ void slapd_daemon( daemon_ports_t *ports ) uniqueIDGenCleanup (); }
if ( ! in_referral_mode ) { /* Close SNMP collator after the plugins closed...
diff --git a/ldap/servers/slapd/plugin.c b/ldap/servers/slapd/plugin.c index d377b33..cc92943 100644 --- a/ldap/servers/slapd/plugin.c +++ b/ldap/servers/slapd/plugin.c @@ -1955,6 +1955,10 @@ plugin_closeall(int close_backends, int close_globals) iterp = dep_plugin_entries; while (iterp) { nextp = iterp->next; + if (!close_backends && (iterp->plugin->plg_type == SLAPI_PLUGIN_DATABASE)) { + iterp = nextp; + continue; + } slapi_entry_free(iterp->e); plugin_free(iterp->plugin); slapi_ch_free((void **)&iterp); }}} Let me reopen this ticket and bug.
attachment 0002-Ticket-48891-ns-slapd-crashes-during-the-shutdown-af.patch
ack
Replying to [comment:12 mreynolds]:
ack +1 note: I also verified valgrind no longer reports Invalid read. Thanks!
Noriko, Mark thanks for this review and for your tests !
'''git merge ticket_48891''' Updating 10949b7..5f7bcdc Fast-forward ldap/servers/slapd/daemon.c | 1 + ldap/servers/slapd/plugin.c | 47 +++++++++++++++++++++++++++++++++++------------ ldap/servers/slapd/proto-slap.h | 1 + 3 files changed, 37 insertions(+), 12 deletions(-)
'''git push origin master''' Counting objects: 8, done. Delta compression using up to 8 threads. Compressing objects: 100% (8/8), done. Writing objects: 100% (8/8), 1.41 KiB | 0 bytes/s, done. Total 8 (delta 6), reused 0 (delta 0) To ssh://git.fedorahosted.org/git/389/ds.git 10949b7..5f7bcdc master -> master
commit 5f7bcdc Author: Thierry Bordaz tbordaz@redhat.com Date: Thu Jun 23 15:48:58 2016 +0200
Metadata Update from @tbordaz: - Issue assigned to tbordaz - Issue set to the milestone: 1.3.5.9
389-ds-base is moving from Pagure to Github. This means that new issues and pull requests will be accepted only in 389-ds-base's github repository.
This issue has been cloned to Github and is available here: - https://github.com/389ds/389-ds-base/issues/1950
If you want to receive further updates on the issue, please navigate to the github issue and click on subscribe button.
subscribe
Thank you for understanding. We apologize for all inconvenience.
Metadata Update from @spichugi: - Issue close_status updated to: wontfix (was: Fixed)
Login to comment on this ticket.