#4042 sssd_krb5_locator: uses discovery even krb5_server specified
Closed: Invalid 4 years ago by jhrozek. Opened 4 years ago by ignatenkobrain.

Hello,

I am running CentOS which has sssd-1.16.2-13.el7_6.8.x86_64 installed.

My sssd.conf looks like:

[sssd]
config_file_version = 2
services = nss, pam, ssh
domains = intgdc.com

[domain/intgdc.com]
cache_credentials = true

# LDAP conf (ID, groups)
id_provider = ldap
ldap_uri = ldap://na3-freeipa02.intgdc.com,ldap://na3-freeipa01.intgdc.com
ldap_search_base = cn=accounts,dc=intgdc,dc=com
ldap_id_use_start_tls = true
ldap_tls_reqcert = demand
ldap_tls_cacert = /etc/pki/ca-trust/source/anchors/ipa_ca.crt
ldap_default_bind_dn = uid=viewer,cn=sysaccounts,cn=etc,dc=intgdc,dc=com
ldap_default_authtok = XXX
ldap_schema = IPA
ldap_user_ssh_public_key = ipaSshPubKey

# Access only for private-cloud
access_provider = simple
simple_allow_groups = team-privatecloud

# KRB5 conf (auth, pass)
auth_provider = krb5
chpass_provider = krb5
krb5_server = na3-freeipa01.intgdc.com,na3-freeipa02.intgdc.com
krb5_realm = INTGDC.COM

From what I understand, if I specify krb5_server, sssd_krb5_locator should not use discovery, but use KDC servers from the list.

But when I run kinit, I get:

[root@ctl-bastion02a ~]# kinit -kt /etc/otpc.keytab otpc/ctl-cml01c.na3.intgdc.com@INTGDC.COM
[12018] 1563034948.39245: Getting initial credentials for otpc/ctl-cml01c.na3.intgdc.com@INTGDC.COM
[12018] 1563034948.39246: Looked up etypes in keytab: aes256-cts, aes128-cts, des3-cbc-sha1, rc4-hmac
[12018] 1563034948.39248: Sending unauthenticated request
[12018] 1563034948.39249: Sending request (214 bytes) to INTGDC.COM
[sssd_krb5_locator] sssd_krb5_locator_init called
[sssd_krb5_locator] open failed [/var/lib/sss/pubconf/kdcinfo.INTGDC.COM][2][No such file or directory].
[sssd_krb5_locator] get_krb5info failed.
[sssd_krb5_locator] sssd_krb5_locator_close called
[12018] 1563034948.39250: Resolving hostname na1-freeipa01.intgdc.com.
[12018] 1563034948.39251: Sending initial UDP request to dgram 108.171.171.249:88
[12018] 1563034949.46678: Resolving hostname na1-freeipa02.intgdc.com.
[12018] 1563034949.46679: Sending initial UDP request to dgram 192.237.128.29:88
[12018] 1563034950.50288: Resolving hostname na1-freeipa01.intgdc.com.
[12018] 1563034950.50289: Resolving hostname na1-freeipa02.intgdc.com.
[12018] 1563034950.50290: Initiating TCP connection to stream 108.171.171.249:88
[12018] 1563034951.93080: Initiating TCP connection to stream 192.237.128.29:88

which means it does not use KDC servers from my list, but uses discovery via DNS. Is this a bug or am I doing something wrong?


I think the important part is here:

[sssd_krb5_locator] open failed [/var/lib/sss/pubconf/kdcinfo.INTGDC.COM][2][No such file or directory].

For some reason, the file is not created.

When SSSD resolves a server internally, it writes its IP address into the kdcinfo file and libkrb5 then reads the file and uses its contents. But since there is nothing to read here, libkrb5 falls back to doing discovery on its own.

Can you enable debugging in the [domain] section and see if something jumps out?

Or, are there any other files matching /var/lib/sss/pubconf/kdcinfo.* created? (I'm trying to see if we have some realm/domain mismatch..)

Or, are there any other files matching /var/lib/sss/pubconf/kdcinfo.* created? (I'm trying to see if we have some realm/domain mismatch..)

No.

[root@ctl-bastion02a ~]# find /var/lib/sss/pubconf/
/var/lib/sss/pubconf/
/var/lib/sss/pubconf/krb5.include.d

Can you enable debugging in the [domain] section and see if something jumps out?

This might be relevant:

(Sun Jul 14 13:33:39 2019) [sssd[be[intgdc.com]]] [krb5_service_new] (0x0100): write_kdcinfo for realm INTGDC.COM set to true
(Sun Jul 14 13:33:39 2019) [sssd[be[intgdc.com]]] [fo_new_service] (0x0400): Creating new service 'KERBEROS'
(Sun Jul 14 13:33:39 2019) [sssd[be[intgdc.com]]] [fo_add_server_to_list] (0x0400): Inserted primary server 'na3-freeipa01.intgdc.com:0' to service 'KERBEROS'
(Sun Jul 14 13:33:39 2019) [sssd[be[intgdc.com]]] [_krb5_servers_init] (0x0400): Added Server na3-freeipa01.intgdc.com
(Sun Jul 14 13:33:39 2019) [sssd[be[intgdc.com]]] [fo_add_server_to_list] (0x0400): Inserted primary server 'na3-freeipa02.intgdc.com:0' to service 'KERBEROS'
(Sun Jul 14 13:33:39 2019) [sssd[be[intgdc.com]]] [_krb5_servers_init] (0x0400): Added Server na3-freeipa02.intgdc.com
(Sun Jul 14 13:33:39 2019) [sssd[be[intgdc.com]]] [krb5_init_kpasswd] (0x0010): Missing krb5_kpasswd option and KDC set explicitly, will use KDC for pasword change operations!
(Sun Jul 14 13:33:39 2019) [sssd[be[intgdc.com]]] [check_lifetime] (0x0200): No lifetime configured.
(Sun Jul 14 13:33:39 2019) [sssd[be[intgdc.com]]] [check_lifetime] (0x0200): No lifetime configured.
(Sun Jul 14 13:33:39 2019) [sssd[be[intgdc.com]]] [sss_krb5_check_options] (0x0100): No kpasswd server explicitly configured, using the KDC or defaults.
(Sun Jul 14 13:33:39 2019) [sssd[be[intgdc.com]]] [parse_krb5_map_user] (0x0100): krb5_map_user is empty!
(Sun Jul 14 13:33:39 2019) [sssd[be[intgdc.com]]] [be_fo_set_srv_lookup_plugin] (0x0400): Trying to set SRV lookup plugin to DNS
(Sun Jul 14 13:33:39 2019) [sssd[be[intgdc.com]]] [fo_set_srv_lookup_plugin] (0x0080): SRV lookup plugin is already set
(Sun Jul 14 13:33:39 2019) [sssd[be[intgdc.com]]] [be_fo_set_srv_lookup_plugin] (0x0080): Unable to set SRV lookup plugin, another plugin may be already in place

If you need more logs, let me know.

Some investigation:

sssm_krb5_initkrb5_init_kdcdp_opt_get_bool(ctx->opts, KRB5_USE_KDCINFO)krb5_service_initkrb5_service_newkrb5_resolve_callback

However, the krb5_resolve_callback is never called. So that means nothing is writing kdcinfo.

@jhrozek I don't know how SSSD works and when that callback should be called... But that seems to be really a problem.

The callback should be called when the name is resolved to IP address. Look for something like:

(Sun Jul 14 21:50:31 2019) [sssd[be[ipa.test]]] [request_watch_destructor] (0x0400): Deleting request watch    
(Sun Jul 14 21:50:31 2019) [sssd[be[ipa.test]]] [set_server_common_status] (0x0100): Marking server 'server.ipa.test' as 'name resolved'
(Sun Jul 14 21:50:31 2019) [sssd[be[ipa.test]]] [be_resolve_server_process] (0x1000): Saving the first resolved server
(Sun Jul 14 21:50:31 2019) [sssd[be[ipa.test]]] [be_resolve_server_process] (0x0200): Found address for server server.ipa.test: [192.168.122.100] TTL 1200
(Sun Jul 14 21:50:31 2019) [sssd[be[ipa.test]]] [ipa_resolve_callback] (0x0400): Constructed uri 'ldap://server.ipa.test'     
(Sun Jul 14 21:50:31 2019) [sssd[be[ipa.test]]] [unique_filename_destructor] (0x2000): Unlinking [/var/lib/sss/pubconf/.krb5info_dummy_2doXTt]
(Sun Jul 14 21:50:31 2019) [sssd[be[ipa.test]]] [unlink_dbg] (0x2000): File already removed: [/var/lib/sss/pubconf/.krb5info_dummy_2doXTt]

I admit the happy-path does not have enough debug statements, so writing the kdcinfo file only emits a message on failure. But if you look around be_resolve_server_process, that's where the file should be created or where you should see an error.

There is 0 set_server in the log file with debug_level=10 =(

Neither there is anything like be_resolve_

Is there any chance you could attach the complete logs? Or feel free to send them to me directly (my fedora nick at redhat dot com).

But I just realised something different. You might need to run authentication via sssd at least once (e.g. run su - user from a non-root or ssh to an IPA user, ...) to trigger resolving the krb5 server. The resolve callback is reactive and only activated as a reaction to needing the krb5 server. Most people use sssd who use Kerberos-anything use SSSD with either AD or IPA where the resolution happens sort of implicitly because sssd needs to authenticate to the LDAP server with GSSAPI, which triggers KDC resolution and also this happens on startup becuase some periodical tasks are run after startup.

So tl;dr, please make sure the logs capture a run after sssd needed to resolve some KDC and either attach them here or send them to me.

I've stopped sssd, cleaned up logs, started sssd, ssh to the machine under my user name and then dumped logs. Sent to you via email.

Just in case you are curious about pam config:

auth        required      pam_env.so
# OTP & pam_sss isn't required for root login (uberkey)
auth        [success=3 default=ignore] pam_succeed_if.so uid eq 0
# No access without OTP
#auth        optional      pam_exec.so /usr/local/bin/get_otp.sh
auth        [success=done default=die] pam_google_authenticator.so secret=/var/local/otpc/${USER}@INTGDC.COM
# Root user and valid OTP should reach this point
auth        sufficient    pam_unix.so nullok use_first_pass
auth        required      pam_deny.so

# Skip pam_sss check for root user (uberkey)
account     [success=1 default=ignore] pam_succeed_if.so uid eq 0
# pam_sss success is a must
account     requisite     pam_sss.so
account     required      pam_permit.so

password    requisite     pam_pwquality.so try_first_pass local_users_only retry=3 authtok_type=
password    sufficient    pam_unix.so sha512 shadow nullok try_first_pass use_authtok
password    sufficient    pam_sss.so use_authtok
password    required      pam_deny.so

session     optional      pam_keyinit.so revoke
session     required      pam_limits.so
-session     optional      pam_systemd.so
session     optional      pam_oddjob_mkhomedir.so umask=0077
session     [success=1 default=ignore] pam_succeed_if.so service in crond quiet use_uid
session     required      pam_unix.so
session     optional      pam_sss.so

Turns out this was happening because pam_sss was not used during auth.

Metadata Update from @jhrozek:
- Issue close_status updated to: Invalid
- Issue status updated to: Closed (was: Open)

4 years ago

SSSD is moving from Pagure to Github. This means that new issues and pull requests
will be accepted only in SSSD's github repository.

This issue has been cloned to Github and is available here:
- https://github.com/SSSD/sssd/issues/5011

If you want to receive further updates on the issue, please navigate to the github issue
and click on subscribe button.

Thank you for understanding. We apologize for all inconvenience.

Login to comment on this ticket.

Metadata