#3948 infinite loop of Kerberos pre-auth errors againt AD leads to user lockout
Closed: cloned-to-github 3 years ago by pbrezina. Opened 5 years ago by praines.

I have experienced this issue with sssd 1.16.2 (CentOS7) and 2.0.0 (Fedora 29)

I have a fairly weird setup in sssd where I need to password authenticate against the corporate AD server but accounts/UIDs do not match between our Linux systems and AD. I have an OpenLDAP server running on a Linux server and a user record will look like

dn: uid=raines,ou=People,dc=linuxdom,dc=org
uid: raines
uidNumber: 5829
gidNumber: 5829
userPassword:: {SASL}per2
carLicense: per2@CORPDOM.ORG

where as you will see I use the carLicense field as a way to remap the username in
for the kerberos auth lookup to the AD principal. UID/GID then does not matter.

My sssd.conf looks like

[sssd]

services = nss, pam
config_file_version = 2
domains = corpdom, linuxdom

[domain/corpdom]

id_provider = ldap
auth_provider = krb5
chpass_provider = none

min_id = 100
dns_discovery_domain = CORPDOM.ORG
krb5_realm = CORPDOM.ORG
krb5_renewable_lifetime = 7d
krb5_lifetime = 10h
krb5_renew_interval = 1h

ldap_user_principal = carLicense
ldap_force_upper_case_realm = True
ldap_id_use_start_tls = True
ldap_search_base = dc=linuxdom,dc=org
ldap_user_search_base = ou=People,dc=linuxdom,dc=org?subtree?(carLicense=*CORPDOM.ORG)
ldap_uri = ldap://ldap2.linuxdom.org, ldap://ldap3.linuxdom.org
ldap_tls_cacertdir = /etc/openldap/cacerts
ldap_tls_reqcert = never
cache_credentials = False
entry_cache_timeout = 600

[domain/linuxdom]

id_provider = ldap
auth_provider = ldap
chpass_provider = none

min_id = 100
ldap_id_use_start_tls = True
cache_credentials = False
ldap_search_base = dc=linuxdom,dc=org
ldap_uri = ldap://ldap2.linuxdom.org, ldap://ldap3.linuxdom.org
ldap_tls_cacertdir = /etc/openldap/cacerts
ldap_tls_reqcert = never
entry_cache_timeout = 600

[nss]

[pam]

This normally works great. However frequently we are getting users locked out on the AD server due to excessive login failures. I had the corporate guys running the AD server send the logs and what we see there are #4771 "Kerberos pre-authentication failed" errors coming from the Linux client running sssd every six seconds. This continues forever constantly locking the users account on the AD server. As soon as I restart sssd on the Linux client it stops.

It seems to be triggered on entering an incorrect password on login or the screensaver, but not consistently. In fact it is pretty rare. I have not been able to force it happen. But I usually see a case of this once or twice a week. My guess is the trigger also involves something with cached credentials or ticket renewal timing but I really have no clue.


I have second issue invovling sss_cache while testing that I am guessing is the same as issue #2760 (sss_cache & negative cache - improve behaviour )

In testing I changed the carLicense field on a user to "NONE". Then later I changed it back to their AD principal name. But sssd refused to do a kerberos authentication on the user still even after doing 'sss_cache -E'. I had to stop sssd, remove all cache files in /var/lib/sss/db and and restart sssd to get it see the change.

I have second issue invovling sss_cache while testing that I am guessing is the same as issue #2760 (sss_cache & negative cache - improve behaviour )

I think this ticket is not related.

In testing I changed the carLicense field on a user to "NONE". Then later I changed it back to their AD principal name. But sssd refused to do a kerberos authentication on the user still even after doing 'sss_cache -E'. I had to stop sssd, remove all cache files in /var/lib/sss/db and and restart sssd to get it see the change.

This might be a side effect of the failed authentication. The 'every six seconds' sounds a bit like the default timeout for Kerberos authentication. A timeout during authentication might switch SSSD into offline mode. In offline mode SSSD will not try to connect to an LDAP server to update the user data for about a minute and hence will not refresh the cached data even if they are expired.

Can you rerun the test and check with 'sssctl domain-status ....' if SSSD is online before trying a new authentication attempt?

bye,
Sumit

This might be a side effect of the failed authentication. The 'every six seconds' sounds a bit like the default timeout for Kerberos authentication. A timeout during authentication might switch SSSD into offline mode. In offline mode SSSD will not try to connect to an LDAP server to update the user data for about a minute and hence will not refresh the cached data even if they are expired.
Can you rerun the test and check with 'sssctl domain-status ....' if SSSD is online before trying a new authentication attempt?

I reran the test and same result. sssctl never showed the domain offline. I would be very surprised if it ever went offline as our network and AD servers should be very solid. I ran 'ssh_cache -E' several times and waited 30 minutes, but it still never saw the change. As soon as I did

cd /var/lib/sss/db; systemctl stop sssd ; \rm * ; systemctl start sssd

it saw the change properly and did the kerberos authentication correctly

Thanks

Can you send SSSD debug logs with at least debug_level=9 in the [domain/...] section of sssd.conf?

Can you send SSSD debug logs with at least debug_level=9 in the [domain/...] section of sssd.conf?

Is there a way to get you the logs confidentially? I think there may be too much sensitive info in them.

In doing this test I did something different than before where I gave myself a different password in the local LDAP server than I have on the corporate AD and this let me discover something.

If I set my LDAP entry to be

userPassword:: {crypt}$6$...(password different than in AD)....
carLicense: NONE

I am able to login using the diffrent password. And as expected I don't have a kerberos ticket since the corpdom domain would not work with carLicense==NONE and so falls through to linuxdom domain. So all good.

But now if I change my LDAP entry to have

userPassword: {crypt}...(a different password yet again than before or in AD)...
carLicense: per2@PARTNERS.ORG

On login my AD password does not work and I have to give the new different password set in userPassword to get in (sssd does see THAT change)

I run 'sss_cache -E' and try again but still my AD password does not work and the new password does.

I wait hours. In order to get sssd to see the change to carLicense I have to run

systemctl stop sssd ; /bin/rm /var/lib/sss/db/* ; systemctl start sssd

and then I can login using the AD password of the user specified by carLicense and I get a kerberos ticket

So it looks like the sss cache is not properly handling expiry of what is specified as "ldap_user_principal " in sssd.conf

This is a different issue than my original post though so maybe it should start a new ticket?

Metadata Update from @pbrezina:
- Issue tagged with: Future milestone

4 years ago

SSSD is moving from Pagure to Github. This means that new issues and pull requests
will be accepted only in SSSD's github repository.

This issue has been cloned to Github and is available here:
- https://github.com/SSSD/sssd/issues/4926

If you want to receive further updates on the issue, please navigate to the github issue
and click on subscribe button.

Thank you for understanding. We apologize for all inconvenience.

Metadata Update from @pbrezina:
- Issue close_status updated to: cloned-to-github
- Issue status updated to: Closed (was: Open)

3 years ago

Login to comment on this ticket.

Metadata