At random intervals, the KDC segfaults at ldap_get_entry_controls.
PID: 736854 (krb5kdc) UID: 0 (root) GID: 0 (root) Signal: 6 (ABRT) Timestamp: Tue 2022-09-27 05:29:18 CDT (4 days ago)
Command Line: /usr/sbin/krb5kdc -P /var/run/krb5kdc.pid -w 4 Executable: /usr/sbin/krb5kdc Control Group: /system.slice/krb5kdc.service Unit: krb5kdc.service Slice: system.slice Boot ID: d8b022d9fb8d40beba050e782fb6058b Machine ID: d1a418c31e1e4a8d9feca7f4210978d0 Hostname: XXXX Storage: /var/lib/systemd/coredump/core.krb5kdc.0.d8b022d9fb8d40beba050e782fb6058b.736854.1664274558000000.lz4 (inaccessible) Message: Process 736854 (krb5kdc) of user 0 dumped core.
Stack trace of thread 736854: #0 0x00007f96e8d6437f raise (libc.so.6) #1 0x00007f96e8d4edb5 abort (libc.so.6) #2 0x00007f96e8d4ec89 __assert_fail_base.cold.0 (libc.so.6) #3 0x00007f96e8d5ca76 __assert_fail (libc.so.6) #4 0x00007f96da538518 ldap_get_entry_controls (libldap_r-2.4.so.2) #5 0x00007f96db4a8bf7 ipadb_ldap_deref_results (ipadb.so) #6 0x00007f96db4ae38b ipadb_get_pac (ipadb.so) #7 0x00007f96db4b0784 ipadb_sign_authdata (ipadb.so) #8 0x00005610b4e56e71 handle_authdata (krb5kdc) #9 0x00005610b4e48084 finish_process_as_req (krb5kdc) #10 0x00005610b4e51556 finish_check_padata (krb5kdc) #11 0x00005610b4e53e74 enc_ts_verify (krb5kdc) #12 0x00005610b4e5204f next_padata (krb5kdc) #13 0x00005610b4e48f77 process_as_req (krb5kdc) #14 0x00005610b4e475b8 dispatch (krb5kdc) #15 0x00005610b4e5cf6d process_tcp_connection_read (krb5kdc) #16 0x00007f96e90f55ec verto_fire (libverto.so.1) #17 0x00007f96d2544ff1 event_process_active_single_queue (libevent-2.1.so.6) #18 0x00007f96d2545787 event_base_loop (libevent-2.1.so.6) #19 0x00005610b4e462e9 main (krb5kdc) #20 0x00007f96e8d50493 __libc_start_main (libc.so.6) #21 0x00005610b4e4667e _start (krb5kdc)
The following is found in /var/log/krb5kdc.log at the time of the crash Sep 25 15:56:45 XXXX krb5kdc709128: worker 709130 exited with status 134
A snip from the full stacktrace shows the segfault "error: Cannot access memory at address".
#4 0x00007f96da538518 in ldap_get_entry_controls (ld=ld@entry=0x0, entry=entry@entry=0x5610b6fc57c0, sctrls=sctrls@entry=0x7ffc3830e330) at getentry.c:89 rc = <optimized out> be = {ber_opts = {lbo_valid = -16064, lbo_options = 56171, lbo_debug = 3}, ber_tag = 0, ber_len = 5, ber_usertag = 0, ber_buf = 0x5610b6ee1e34 "", ber_ptr = 0x3b9b563cdd950c00 <error: Cannot access memory at address 0x3b9b563cdd950c00>, ber_end = 0x5610b6ef5890 "\247~\002\f", ber_sos_ptr = 0x7ffc3830e450 "", ber_rwptr = 0x0, ber_memctx = 0x7f96db4a7ea1 <ipadb_need_retry+49>} PRETTY_FUNCTION = "ldap_get_entry_controls"
Unknown. Random crashes of KDC a few times a week.
ipa-server-4.9.3-1.module_el8.5.0+836+f9e1ecc9.x86_64 ipa-client-4.9.3-1.module_el8.5.0+836+f9e1ecc9.x86_64 389-ds-base-1.4.3.23-7.module_el8.5.0+889+90e0384f.x86_64 pki-ca-10.11.0-1.module_el8.5.0+876+d4bb8aa6.noarch krb5-server-1.18.2-13.el8.x86_64
cat /etc/redhat-release CentOS Stream release 8
uname -r 4.18.0-331.el8.x86_64
set = {__val = {0, 94629793542944, 2, 140286128448587, 94630941917184, 94629793542944, 94629793542944, 94629793542944, 94629793542944, 94629793543024, 94629793543044, 94629793542944, 94629793543044, 0, 0, 0}} pid = <optimized out> tid = <optimized out> ret = <optimized out>
#1 0x00007f96e8d4edb5 in __GI_abort () at abort.c:79 save_stage = 1 act = {__sigaction_handler = {sa_handler = 0x5610b73ab320, sa_sigaction = 0x5610b73ab320}, sa_mask = {__val = {0, 140286131882944, 140286129526304, 0, 0, 0, 140721251213800, 21474836480, 140721251213648, 140286129582768, 140286129567384, 0, 4295121489043459072, 140286129552301, 0, 140286129567384}}, sa_flags = -631936836, sa_restorer = 0x7f96da559228} sigs = {__val = {32, 0 <repeats 15 times>}}
at assert.c:92 str = 0x5610b73ab320 "\020\204\030\267\020V" total = 4096
#3 0x00007f96e8d5ca76 in GIassertfail (assertion=assertion@entry=0x7f96da559228 "ld != NULL", file=file@entry=0x7f96da5568bc "getentry.c", line=line@entry=89, function=function@entry=0x7f96da5568e0 <PRETTY_FUNCTION.8960> "ldap_get_entry_controls") at assert.c:101 No locals. #4 0x00007f96da538518 in ldap_get_entry_controls (ld=ld@entry=0x0, entry=entry@entry=0x5610b6fc57c0, sctrls=sctrls@entry=0x7ffc3830e330) at getentry.c:89 rc = <optimized out> be = {ber_opts = {lbo_valid = -16064, lbo_options = 56171, lbo_debug = 3}, ber_tag = 0, ber_len = 5, ber_usertag = 0, ber_buf = 0x5610b6ee1e34 "", ber_ptr = 0x3b9b563cdd950c00 <error: Cannot access memory at address 0x3b9b563cdd950c00>, ber_end = 0x5610b6ef5890 "\247~\002\f", ber_sos_ptr = 0x7ffc3830e450 "", ber_rwptr = 0x0, ber_memctx = 0x7f96db4a7ea1 <ipadb_need_retry+49>} PRETTY_FUNCTION = "ldap_get_entry_controls" #5 0x00007f96db4a8bf7 in ipadb_ldap_deref_results (lcontext=0x0, le=le@entry=0x5610b6fc57c0, results=results@entry=0x7ffc3830e3e0) at ipa_kdb_common.c:572 ctrls = 0x0 derefctrl = 0x0 ret = <optimized out> #6 0x00007f96db4ae38b in ipadb_fill_info3 (info3=0x5610b6f374a0, authtime=1664274523, memctx=0x5610b7403bb0, flags=112, lentry=0x5610b6fc57c0, ipactx=0x5610b6ef5890) at ipa_kdb_mspac.c:669 deref_results = 0x0 ret = <optimized out> is_host = <optimized out> is_service = <optimized out> is_ipauser = <optimized out> sid = {sid_rev_num = 16 '\020', num_auths = -27 '\345', id_auth = "08\374\177\000", sub_auths = {3717532672, 1000035900, 20, 0, 3932682600, 32662, 0, 0, 0, 0, 0, 0, 0, 0, 0}} intres = -491473159 timeres = 1652291979 objectclasses = 0x5610b6fcddd0 c = <optimized out> is_idobject = <optimized out> prigid = 515 strres = 0x5610b721fdf0 "\020n\373\266\020V" is_user = <optimized out> princ = 0x5610b6f48b70 deref_results = <optimized out> sid = <optimized out> prigid = <optimized out> timeres = <optimized out> strres = <optimized out> intres = <optimized out> ret = <optimized out> objectclasses = <optimized out> [60/192534] c = <optimized out> is_host = <optimized out> is_user = <optimized out> is_service = <optimized out> is_ipauser = <optimized out> is_idobject = <optimized out> princ = <optimized out> sep = <optimized out> is_master = <optimized out> dres = <optimized out> dval = <optimized out> gsid = <optimized out> trid = <optimized out> tgid = <optimized out> s = <optimized out> count = <optimized out> #7 ipadb_get_pac (kcontext=kcontext@entry=0x5610b6fb8cc0, client=0x5610b70590d0, flags=112, authtime=1664274523, pac=0x7ffc3830e630) at ipa_kdb_mspac.c:868 tmpctx = 0x5610b7403bb0 ied = <optimized out> ipactx = 0x5610b6ef5890 results = 0x5610b6fc57c0 lentry = 0x5610b6fc57c0 pac_data = {data = 0x5610b6f48b70 "\200\265\f\267\020V", length = 140721251215184} data = {magic = -1223920800, length = 22032, data = 0x7f96ea93b330 ""} pac_info = {logon_info = {info = 0x5610b6f374a0}, credential_info = {version = 3069408416, encryption_type = 22032, encrypted_data = {data = 0x0, length = 0}}, srv_cksum = {type = 3069408416, signature = {data = 0x0, --Type <RET> for more, q to quit, c to continue without paging--c length = 0}}, kdc_cksum = {type = 3069408416, signature = {data = 0x0, length = 0}}, logon_name = {logon_time = 94629788873888, size = 0, account_name = 0x0}, constrained_delegation = {info = 0x5610b6f374a0}, upn_dns_ info = {upn_name_size = 29856, upn_name = 0x0, dns_domain_name_size = 0, dns_domain_name = 0x0, flags = 0}, unknown = {remaining = {data = 0x5610b6f374a0 "\200\377\303\352[\322\330\001\377\377\377\377\377\377\377\177\377\377\377\37 7\377\377\377\177\200\267\372\341e\330\001\200\267\372\341e\330\001\377\377\377\377\377\377\377\177", length = 0}}} kerr = <optimized out> ndr_err = <optimized out> pac_upn = {logon_info = {info = 0x0}, credential_info = {version = 0, encryption_type = 0, encrypted_data = {data = 0x3b9b563cdd950c00 <error: Cannot access memory at address 0x3b9b563cdd950c00>, length = 109044118360876996 98}}, srv_cksum = {type = 0, signature = {data = 0x3b9b563cdd950c00 <error: Cannot access memory at address 0x3b9b563cdd950c00>, length = 10904411836087699698}}, kdc_cksum = {type = 0, signature = {data = 0x3b9b563cdd950c00 <error: Cannot access memory at address 0x3b9b563cdd950c00>, length = 10904411836087699698}}, logon_name = {logon_time = 0, size = 3072, account_name = 0x975440ca769288f2 <error: Cannot access memory at address 0x975440ca769288f2>}, const rained_delegation = {info = 0x0}, upn_dns_info = {upn_name_size = 0, upn_name = 0x3b9b563cdd950c00 <error: Cannot access memory at address 0x3b9b563cdd950c00>, dns_domain_name_size = 35058, dns_domain_name = 0xa <error: Cannot acce ss memory at address 0xa>, flags = 3069479792}, unknown = {remaining = {data = 0x0, length = 4295121489043459072}}} principal = 0x0 #8 0x00007f96db4b0784 in ipadb_sign_authdata (context=0x5610b6fb8cc0, flags=112, client_princ=<optimized out>, client=0x5610b70590d0, server=0x5610b7151eb0, krbtgt=0x5610b7151eb0, client_key=0x5610b6f45f70, server_key=0x5610b6f45f 58, krbtgt_key=0x5610b6f45f40, session_key=0x5610b6f46070, authtime=1664274523, tgt_auth_data=0x0, signed_auth_data=0x7ffc3830e890) at ipa_kdb_mspac.c:2285 ks_client_princ = 0x5610b6f45e40 pac_auth_data = 0x0 authdata = {0x0, 0x0} ad = {magic = -1226383344, ad_type = 22032, length = 64, contents = 0xfffffffffffffea8 <error: Cannot access memory at address 0xfffffffffffffea8>} is_as_req = 1 kerr = <optimized out> pac = 0x0 pac_data = {magic = 16, length = 0, data = 0x6e00000002 <error: Cannot access memory at address 0x6e00000002>} ipactx = <optimized out> with_pac = true with_pad = false make_ad = <optimized out> result = -344 client_entry = 0x5610b70590d0 is_equal = <optimized out> force_reinit_mspac = <optimized out> #9 0x00005610b4e56e71 in fetch_kdb_authdata (req=0x5610b701dae0, req=0x5610b701dae0, auth_indicators=0x5610b6f460d0, enc_tkt_reply=0x5610b6f45e68, enc_tkt_req=0x0, ad_info=0x0, altcprinc=0x0, local_tgt_key=0x5610b6f45f40, header_k ey=0x0, server_key=0x5610b6f45f58, client_key=0x5610b6f45f70, local_tgt=0x5610b7151eb0, header_server=0x0, server=0x5610b7151eb0, client=0x5610b70590d0, flags=112, context=0x5610b6fb8cc0) at kdc_authdata.c:366 ret = <optimized out> tgt_authdata = <optimized out> db_authdata = 0x0 tgs_req = <optimized out> actual_client = <optimized out> ret = <optimized out> tgt_authdata = <optimized out> db_authdata = <optimized out> tgs_req = <optimized out> actual_client = <optimized out> PRETTY_FUNCTION = "fetch_kdb_authdata" #10 handle_authdata (context=0x5610b6fb8cc0, flags=112, client=0x5610b70590d0, server=0x5610b7151eb0, subject_server=subject_server@entry=0x0, local_tgt=0x5610b7151eb0, local_tgt_key=0x5610b6f45f40, client_key=0x5610b6f45f70, server_key=0x5610b6f45f58, subject_key=0x0, req_pkt=0x5610b6ef30b0, req=0x5610b701dae0, altcprinc=0x0, ad_info=0x0, enc_tkt_req=0x0, auth_indicators=0x5610b6f460d0, enc_tkt_reply=0x5610b6f45e68) at kdc_authdata.c:854 h = <optimized out> ret = <optimized out> i = <optimized out> #11 0x00005610b4e48084 in finish_process_as_req (state=0x5610b6f45e30, errcode=<optimized out>) at do_as_req.c:284 server_key = 0x5610b6fb2780 as_encrypting_key = 0x0 response = 0x0 emsg = 0x0 did_log = 0 oldrespond = 0x5610b4e472a0 <finish_dispatch_cache> oldarg = 0x5610b6ee0720 kdc_active_realm = 0x5610b6ef5b50 au_state = 0x5610b7037d00 PRETTY_FUNCTION = "finish_process_as_req" #12 0x00005610b4e51556 in finish_check_padata (state=0x5610b7537730, code=<optimized out>) at kdc_preauth.c:1197 respond = 0x5610b4e48440 <finish_preauth> arg = 0x5610b6f45e30 #13 0x00005610b4e53e74 in enc_ts_verify (context=0x5610b6fb8cc0, req_pkt=<optimized out>, request=<optimized out>, enc_tkt_reply=0x5610b6f45e68, pa=<optimized out>, cb=<optimized out>, rock=0x5610b6f45fb8, moddata=0x0, respond=0x5610b4e520d0 <finish_verify_padata>, arg=0x5610b7537730) at kdc_preauth_encts.c:118 pa_enc = 0x5610b717fb20 retval = 0 scratch = {magic = -384887408, length = 66, data = 0x5610b6fcde30 "0@\240\003\002\001\022\242\071\004\067\020ϒ_d\342\223q,L*D\371\a\226Z\365]\316\347\003\260\341\244i¨Q\324q\203\220\Q\033ڿp\215\244\031qME\031\nM\372\071A\327(B\367G"} enc_ts_data = {magic = -1225028416, length = 27, data = 0x0} enc_data = 0x5610b70591b0 key = {magic = -1760647421, enctype = 18, length = 0, contents = 0x0} client_key = 0x5610b7058d20 start = 1 #14 0x00005610b4e5204f in next_padata (state=<optimized out>) at kdc_preauth.c:1308 PRETTY_FUNCTION = "next_padata" #15 0x00005610b4e48f77 in process_as_req (request=<optimized out>, req_pkt=req_pkt@entry=0x5610b6ef30b0, local_addr=local_addr@entry=0x5610b6ef3098, remote_addr=remote_addr@entry=0x5610b704f650, kdc_active_realm=0x5610b6ef5b50, --Type <RET> for more, q to quit, c to continue without paging-- vctx=vctx@entry=0x5610b6f14f30, respond=0x5610b4e472a0 <finish_dispatch_cache>, arg=0x5610b6ee0720) at do_as_req.c:797 errcode = <optimized out> s_flags = <optimized out> encoded_req_body = {magic = 421, length = 156, data = 0x5610b776e701 "0\201\231\240\a\003\005"} useenctype = <optimized out> state = 0x5610b6f45e30 au_state = 0x5610b7037d00 #16 0x00005610b4e475b8 in dispatch (cb=0x5610b5065140 <shandle>, local_addr=local_addr@entry=0x5610b6ef3098, remote_addr=remote_addr@entry=0x5610b704f650, pkt=pkt@entry=0x5610b6ef30b0, is_tcp=is_tcp@entry=1, vctx=vctx@entry=0x5610b6f14f30, respond=0x5610b4e5a850 <process_tcp_response>, arg=0x5610b6ef3000) at dispatch.c:201 retval = 0 req = 0x5610b701dae0 response = 0x0 state = 0x5610b6ee0720 handle = <optimized out> kdc_err_context = 0x5610b6fb8cc0 #17 0x00005610b4e5cf6d in process_tcp_connection_read (ctx=0x5610b6f14f30, ev=<optimized out>) at net-server.c:1359 local_saddrlen = 16 state = 0x5610b6ef3000 conn = 0x5610b704f560 nread = <optimized out> len = 425 #18 0x00007f96e90f55ec in verto_fire () from /lib64/libverto.so.1 No symbol table info available. #19 0x00007f96d2544ff1 in event_process_active_single_queue () from /lib64/libevent-2.1.so.6 No symbol table info available. #20 0x00007f96d2545787 in event_base_loop () from /lib64/libevent-2.1.so.6 No symbol table info available. #21 0x00005610b4e462e9 in main (argc=5, argv=0x7ffc3830f0f8) at main.c:1064 retval = 0 kcontext = 0x5610b6e6f610 realm = <optimized out> ctx = 0x5610b6f14f30 tcp_listen_backlog = 5 i = <optimized out>
e\330\001\200\267\372\341
This is duplicate of https://pagure.io/freeipa/issue/9251 I am closing this one to keep discussion in one place.
Metadata Update from @abbra: - Issue close_status updated to: duplicate - Issue status updated to: Closed (was: Open)
I have another core dump. Where would you like that sent?
On Sat, Oct 1, 2022, 11:13 PM Alexander Bokovoy pagure@pagure.io wrote:
abbra added a new comment to an issue you are following: This is duplicate of https://pagure.io/freeipa/issue/9251 I am closing this one to keep discussion in one place. To reply, visit the link below or just reply to this email https://pagure.io/freeipa/issue/9253
abbra added a new comment to an issue you are following: This is duplicate of https://pagure.io/freeipa/issue/9251 I am closing this one to keep discussion in one place.
To reply, visit the link below or just reply to this email https://pagure.io/freeipa/issue/9253
please send it abokovoy@ redhat.com.
Login to comment on this ticket.