ipa-server-install crashed as DS crashed:
# ipa-server-install ... Global DNS configuration in LDAP server is empty You can use 'dnsconfig-mod' command to set global DNS options that would override settings in local named.conf files Restarting the web server Unexpected error - see /var/log/ipaserver-install.log for details: CalledProcessError: Command ''/bin/systemctl' 'restart' 'ipa.service'' returned non-zero exit status 1 # ipactl restart Starting Directory Service Starting krb5kdc Service Starting kadmin Service Starting named Service Starting ipa_memcached Service Starting httpd Service Starting pki-tomcatd Service Failed to start pki-tomcatd Service Shutting down Aborting ipactl # systemctl status dirsrv@MKOSEK-FEDORA20-TEST.service dirsrv@MKOSEK-FEDORA20-TEST.service - 389 Directory Server MKOSEK-FEDORA20-TEST. Loaded: loaded (/usr/lib/systemd/system/dirsrv@.service; enabled) Active: failed (Result: signal) since Fri 2014-09-05 14:29:30 CEST; 6min ago Process: 2744 ExecStopPost=/bin/rm -f /var/run/dirsrv/slapd-%i.pid (code=exited, status=0/SUCCESS) Process: 2653 ExecStart=/usr/sbin/ns-slapd -D /etc/dirsrv/slapd-%i -i /var/run/dirsrv/slapd-%i.pid -w /var/run/dirsrv/slapd-%i.startpid (code=exited, status=0/SUCCESS) Main PID: 2676 (code=killed, signal=SEGV) Sep 05 14:29:27 ipa.mkosek-fedora20.test systemd[1]: Started 389 Directory Server MKOSEK-FEDORA20-TEST.. Sep 05 14:29:29 ipa.mkosek-fedora20.test ns-slapd[2676]: GSSAPI server step 1 Sep 05 14:29:29 ipa.mkosek-fedora20.test ns-slapd[2676]: GSSAPI server step 2 Sep 05 14:29:29 ipa.mkosek-fedora20.test ns-slapd[2676]: GSSAPI server step 3 Sep 05 14:29:29 ipa.mkosek-fedora20.test ns-slapd[2676]: GSSAPI server step 1 Sep 05 14:29:29 ipa.mkosek-fedora20.test ns-slapd[2676]: GSSAPI server step 2 Sep 05 14:29:29 ipa.mkosek-fedora20.test ns-slapd[2676]: GSSAPI server step 3 Sep 05 14:29:30 ipa.mkosek-fedora20.test systemd[1]: dirsrv@MKOSEK-FEDORA20-TEST.service: main pro...EGV Sep 05 14:29:30 ipa.mkosek-fedora20.test systemd[1]: Unit dirsrv@MKOSEK-FEDORA20-TEST.service ente...te. Hint: Some lines were ellipsized, use -l to show in full.
Versions:
freeipa-server-4.0.1GIT5de8d88-0.fc20.x86_64 389-ds-base-1.3.2.23-1.fc20.x86_64 pki-ca-10.1.1-1.fc20.noarch
attachment stacktrace.1409920931.txt.gz
Glad you could reproduce it, Thierry! Did you have a chance to run DS via valgrind?
The crash occurs because the modified entry (in pblock) is corrupted. When processing a MOD, the target entry is duplicated and mod_ops are processed on the duplicated entry. This duplicated entry (SLAPI_MODIFY_EXISTING_ENTRY) is corrupted (freed ??) when the be-txn-pre-op (dna_be_txn_pre_op) is called. The only possibilities I can think of are: - one of the preop/txnpreop frees that entry - There is a heap corruption before that this function is called and this thread is victim of this corruption.
Crashing thread is the only active thread at this time
attachment 0001-Ticket-47889-DS-crashed-during-ipa-server-install-on.patch
Pushed to master on behalf of Thierry (Thank you, Thierry!!) 0f1a203..3b5f3fa master -> master commit 3b5f3fa
Pushed to 389-ds-base-1.3.3 branch: cad5b96..0363fa4 389-ds-base-1.3.3 -> 389-ds-base-1.3.3 commit 0363fa4
Pushed to 389-ds-base-1.3.2 branch: 0056b61..1db611e 389-ds-base-1.3.2 -> 389-ds-base-1.3.2 commit 1db611e
I leave this ticket opened. Thierry, if the fix is verified, please close it. Thank you!!
Crash with 389-ds-base-1.3.3.2-1.fc20.x86_64 stacktrace.1410511113.txt.gz
I tested with 389-ds-base-1.3.3.2-1.fc20.x86_64 (rebuilt F21 SRPM from Fedora) and I still got a crash - see attached stacktrace.
This is the same crash. What is weird is that the source is not containing the fix (setting of SLAPI_MODIFY_EXISTING_ENTRY is not present) :
{{{ (gdb) l 520,535 520 / reset ec set cache in id2entry_add_ext / 521 if (ec) { 522 / must duplicate ec before returning it to cache, 523 * which could free the entry. / 524 if ( (tmpentry = backentry_dup( ec )) == NULL ) { 525 ldap_result_code= LDAP_OPERATIONS_ERROR; 526 goto error_return; 527 } 528 if (cache_is_in_cache(&inst->inst_cache, ec)) { 529 CACHE_REMOVE(&inst->inst_cache, ec); 530 } 531 CACHE_RETURN(&inst->inst_cache, &ec); 532 ec = original_entry; 533 original_entry = tmpentry; 534 tmpentry = NULL; 535 }
}}}
However the branch 1.3.3 contains the fix and the version 1.3.3.2 contains it.
It is looking that the test was done from http://koji.fedoraproject.org/koji/buildinfo?buildID=576829 (389-ds-base-1.3.3.2-1.fc21.src.rpm) but this build is missing some of the fixes (dna crash 47889 and possibly some part of 47885).
The source tarball for 389-ds-base-1.3.3.2-1 was incorrect. I have corrected this in 389-ds-base-1.3.3.2-2:
http://koji.fedoraproject.org/koji/taskinfo?taskID=7572257
I have successfully tested copr builds of master branch and 1.3.2. (I needed to backport Ludwig fix for freeipa Update-SSL-ciphers-configured-in-389-ds-base to configure the instance).
freeipa (+dns) restart did not trigger crash after more than 100 restarts.
Metadata Update from @mkosek: - Issue assigned to tbordaz - Issue set to the milestone: 1.3.2.24
389-ds-base is moving from Pagure to Github. This means that new issues and pull requests will be accepted only in 389-ds-base's github repository.
This issue has been cloned to Github and is available here: - https://github.com/389ds/389-ds-base/issues/1220
If you want to receive further updates on the issue, please navigate to the github issue and click on subscribe button.
subscribe
Thank you for understanding. We apologize for all inconvenience.
Metadata Update from @spichugi: - Issue close_status updated to: wontfix (was: Fixed)
Login to comment on this ticket.