#2906 Cannot retrieve users after upgrade from 1.12 to 1.13
Closed: Fixed None Opened 3 years ago by jayhendren.

In my organization, we're retrieving user/group information from Active Directory using the LDAP provider in SSSD.

(Since I know it's going to come up - we can't use the AD provider for technical + political reasons: my organization doesn't control the AD service we're using, and we don't have privileges to create host principals in the domain, which means we can't use winbind to join the domain; instead we just use simple LDAP queries to retreive user information. Authentication is done using a separate MIT Kerberos domain (don't ask... also a complicated story). Also, we used to support RHEL 5, which didn't even have the AD provider.)

After upgrading from SSSD 1.12 to 1.13 on RHEL/CentOS 7, user lookups have stopped working (but group lookups still work).

Following is what I see on test VMs that I've spun up to demonstrate the issue.

sssd.conf:

[sssd]
debug_level = 6
config_file_version = 2
services = nss, pam
domains = LDAP

[nss]
filter_users = root,named,avahi,haldaemon,dbus,radiusd,news,nscd

[pam]

[domain/LDAP]
debug_level = 9
id_provider = ldap
auth_provider = krb5
chpass_provider = krb5
sudo_provider = ldap
enumerate = false
cache_credentials = false
ldap_schema = rfc2307bis
ldap_uri = ldaps://ad.myuniversity.edu:636
ldap_search_base = dc=ad,dc=myuniversity,dc=edu
ldap_user_search_base = DC=ad,DC=myuniversity,DC=edu
ldap_user_object_class = user
ldap_user_name = sAMAccountName
override_homedir = 
shell_fallback = /bin/bash
ldap_group_search_base = OU=Unix Groups,OU=OIT - UnixOps,OU=SIS,OU=ITS,OU=Departments Schools and Colleges,DC=ad,DC=myuniversity,DC=edu
ldap_group_object_class = group
ldap_group_name = extensionAttribute15
ldap_id_use_start_tls = true
ldap_tls_reqcert = never
ldap_tls_cacert = /etc/pki/tls/certs/ca-bundle.crt
ldap_default_bind_dn = CN=oit-sac,ou=service accounts,dc=ad,dc=myuniversity,dc=edu
ldap_default_authtok = redacted
access_provider = ldap
ldap_access_filter = sAMAccountName=configman-test
min_id = 1
max_id = 0
krb5_realm = LOCAL
krb5_server = localhost
schema = rfc2307bis
ldap_user_gecos = displayName
ldap_user_home_directory = unixHomeDirectory

SSSD 1.13.0 on CentOS 7:

[vagrant@default-centos-71 ~]$ rpm -q sssd
sssd-1.13.0-40.el7_2.1.x86_64
[vagrant@default-centos-71 ~]$ date; sudo sss_cache -E
Mon Dec 21 18:56:01 UTC 2015
[vagrant@default-centos-71 ~]$ date; getent passwd configman-test
Mon Dec 21 18:56:09 UTC 2015
[vagrant@default-centos-71 ~]$ date; getent group facstaff
Mon Dec 21 19:04:17 UTC 2015
facstaff:*:829:

(sssd_LDAP.log attached)

I found it rather challenging to downgrade from SSSD 1.13 to 1.12 or 1.11 on CentOS 7, and I don't want to go through that effort again in order to obtain debug output, so instead here is a working example using SSSD 1.12 on CentOS 6 with an identical sssd.conf:

[vagrant@default-centos-66 ~]$ rpm -q sssd
sssd-1.12.4-47.el6_7.4.x86_64
[vagrant@default-centos-66 ~]$ date; sudo sss_cache -E
Mon Dec 21 19:02:07 UTC 2015
[vagrant@default-centos-66 ~]$ date; getent passwd configman-test
Mon Dec 21 19:02:16 UTC 2015
configman-test:*:451737:96:Configuration Management:/home/configman-test:/bin/bash
[vagrant@default-centos-66 ~]$ date; getent group facstaff
Mon Dec 21 19:02:59 UTC 2015
facstaff:*:829:
[vagrant@default-centos-66 ~]$

oh, and ldapsearch works fine from both VMs using the same base and filter as SSSD uses:

[vagrant@default-centos-71 ~]$ ldapsearch -H ldaps://ad.myuniversity.edu -b DC=ad,DC=myuniversity,DC=edu -x -D 'CN=oit-sac,ou=service accounts,dc=ad,dc=myuniversity,dc=edu' -w 'redacted' '(&(sAMAccountName=configman-test)(objectclass=user)(sAMAccountName=*)(&(uidNumber=*)(!(uidNumber=0))))' cn
# extended LDIF
#
# LDAPv3
# base <DC=ad,DC=myuniversity,DC=edu> with scope subtree
# filter: (&(sAMAccountName=configman-test)(objectclass=user)(sAMAccountName=*)(&(uidNumber=*)(!(uidNumber=0))))
# requesting: cn 
#

# configman-test, People, ad.myuniversity.edu
dn: CN=configman-test,OU=People,DC=ad,DC=myuniversity,DC=edu
cn: configman-test

# search reference
ref: ldaps://ad.myuniversity.edu/CN=Configuration,DC=ad,DC=myuniversity,DC=edu

# search result
search: 2
result: 0 Success

# numResponses: 3
# numEntries: 1
# numReferences: 1
[vagrant@default-centos-71 ~]$



[vagrant@default-centos-66 ~]$ ldapsearch -H ldaps://ad.myuniversity.edu -b DC=ad,DC=myuniversity,DC=edu -x -D 'CN=oit-sac,ou=service accounts,dc=ad,dc=myuniversity,dc=edu' -w 'redacted' '(&(sAMAccountName=configman-test)(objectclass=user)(sAMAccountName=*)(&(uidNumber=*)(!(uidNumber=0))))' cn
# extended LDIF
#
# LDAPv3
# base <DC=ad,DC=myuniversity,DC=edu> with scope subtree
# filter: (&(sAMAccountName=configman-test)(objectclass=user)(sAMAccountName=*)(&(uidNumber=*)(!(uidNumber=0))))
# requesting: cn 
#

# configman-test, People, ad.myuniversity.edu
dn: CN=configman-test,OU=People,DC=ad,DC=myuniversity,DC=edu
cn: configman-test

# search reference
ref: ldaps://ad.myuniversity.edu/CN=Configuration,DC=ad,DC=myuniversity,DC=edu

# search result
search: 2
result: 0 Success

# numResponses: 3
# numEntries: 1
# numReferences: 1

The backend logs do not show an issue, can you attached the sssd_nss.log file as well.

Fields changed

cc: => sbose

Sure thing. I've cranked debug_level up to 9 for nss and attached the logs after repeating the steps mentioned above.

Replying to [comment:1 jayhendren]:

oh, and ldapsearch works fine from both VMs using the same base and filter as SSSD uses:

{{{
[vagrant@default-centos-71 ~]$ ldapsearch -H ldaps://ad.myuniversity.edu -b DC=ad,DC=myuniversity,DC=edu -x -D 'CN=oit-sac,ou=service accounts,dc=ad,dc=myuniversity,dc=edu' -w 'redacted' '(&(sAMAccountName=configman-test)(objectclass=user)(sAMAccountName=)(&(uidNumber=)(!(uidNumber=0))))' cn

extended LDIF

LDAPv3

base <dc=ad,dc=myuniversity,dc=edu> with scope subtree

filter: (&(sAMAccountName=configman-test)(objectclass=user)(sAMAccountName=)(&(uidNumber=)(!(uidNumber=0))))

requesting: cn

configman-test, People, ad.myuniversity.edu

dn: CN=configman-test,OU=People,DC=ad,DC=myuniversity,DC=edu
cn: configman-test

search reference

ref: ldaps://ad.myuniversity.edu/CN=Configuration,DC=ad,DC=myuniversity,DC=edu

search result

search: 2
result: 0 Success

numResponses: 3

numEntries: 1

numReferences: 1

[vagrant@default-centos-71 ~]$
}}}

That's really weird because te same ldapsearch failed in sssd.logs

[be_get_account_info] (0x0200): Got request for [0x1001][1][name=configman-test]

[sdap_search_user_next_base] (0x0400): Searching for users with base [DC=ad,DC=myuniversity,DC=edu]
[sdap_get_generic_ext_step] (0x0400): calling ldap_search_ext with [(&(sAMAccountName=configman-test)(objectclass=user)(sAMAccountName=*)(&(uidNumber=*)(!(uidNumber=0))))][DC=ad,DC=myuniversity,DC=edu].

[sdap_op_add] (0x2000): New operation 6 timeout 6
[sdap_process_result] (0x2000): Trace: sh[0x7f0862b4ade0], connected[1], ops[0x7f0863450f30], ldap[0x7f0862b4be00]
[sdap_process_message] (0x4000): Message type: [LDAP_RES_SEARCH_ENTRY]

[sdap_parse_entry] (0x1000): OriginalDN: [CN=configman-test,OU=People,DC=ad,DC=myuniversity,DC=edu].
[sdap_parse_range] (0x2000): No sub-attributes for [objectClass]
[sdap_parse_range] (0x2000): No sub-attributes for [cn]
[sdap_parse_range] (0x2000): No sub-attributes for [displayName]
[sdap_parse_range] (0x2000): No sub-attributes for [memberOf]
[sdap_parse_range] (0x2000): No sub-attributes for [uSNChanged]
[sdap_parse_range] (0x2000): No sub-attributes for [userAccountControl]
[sdap_parse_range] (0x2000): No sub-attributes for [accountExpires]
[sdap_parse_range] (0x2000): No sub-attributes for [sAMAccountName]
[sdap_parse_range] (0x2000): No sub-attributes for [modifyTimeStamp]
[sdap_parse_range] (0x2000): No sub-attributes for [gidNumber]
[sdap_parse_range] (0x2000): No sub-attributes for [uidNumber]
[sdap_parse_range] (0x2000): No sub-attributes for [loginShell]
[sdap_parse_range] (0x2000): No sub-attributes for [unixHomeDirectory]

[sdap_process_result] (0x2000): Trace: sh[0x7f0862b4ade0], connected[1], ops[0x7f0863450f30], ldap[0x7f0862b4be00]
[sdap_process_message] (0x4000): Message type: [LDAP_RES_SEARCH_REFERENCE]
[sdap_get_generic_ext_add_references] (0x1000): Additional References: ldaps://ad.myuniversity.edu/CN=Configuration,DC=ad,DC=myuniversity,DC=edu
[sdap_process_result] (0x2000): Trace: sh[0x7f0862b4ade0], connected[1], ops[0x7f0863450f30], ldap[0x7f0862b4be00]
[sdap_process_result] (0x2000): Trace: ldap_result found nothing!
[sdap_process_result] (0x2000): Trace: sh[0x7f0862b4ade0], connected[1], ops[0x7f0863450f30], ldap[0x7f0862b4be00]
[sdap_process_message] (0x4000): Message type: [LDAP_RES_SEARCH_REFERENCE]
[sdap_get_generic_ext_add_references] (0x1000): Additional References: ldaps://ad.myuniversity.edu/CN=Schema,CN=Configuration,DC=ad,DC=myuniversity,DC=edu
[sdap_process_result] (0x2000): Trace: sh[0x7f0862b4ade0], connected[1], ops[0x7f0863450f30], ldap[0x7f0862b4be00]
[sdap_process_result] (0x2000): Trace: ldap_result found nothing!
[sdap_process_result] (0x2000): Trace: sh[0x7f0862b4ade0], connected[1], ops[0x7f0863450f30], ldap[0x7f0862b4be00]
[sdap_process_message] (0x4000): Message type: [LDAP_RES_SEARCH_RESULT]
[sdap_get_generic_op_finished] (0x0400): Search result: Success(0), no errmsg set
[sdap_op_destructor] (0x2000): Operation 6 finished

[sdap_get_users_done] (0x0040): Failed to retrieve users
[sdap_id_op_done] (0x4000): releasing operation connection
[acctinfo_callback] (0x0100): Request processed. Returned 3,22,User lookup failed
[sdap_process_result] (0x2000): Trace: sh[0x7f0862b4ade0], connected[1], ops[(nil)], ldap[0x7f0862b4be00]
[sdap_process_result] (0x2000): Trace: ldap_result found nothing!

BTW I'm little bit confused. You mentioned that you cannot retrievee users after upgrade from 1.12 to 1.13 but you attached log files from CentOS 7.1 which has sssd-1.12.x. And BTW CentOS has sssd-1.11.x.

So does it work for you on CentOS 6.7 with the same configuration file?

Is there a problem on CentOS 7.1 (sssd-1.12.x) or CentOS 7.2 (1.13.x)?

Could you test on CentOS 6.7 + sssd-1.13.x from extra repo https://copr.fedoraproject.org/coprs/lslebodn/sssd-1-13/ ?

cc: sbose => sbose, lslebodn

It looks like the search is successful but the processing of the referral information cause the request to fail. So I guess patch c9db9d3 "LDAP: Support returning referral information" or related ones might cause the issue.

Nevertheless the referrals are not needed here, can you check if adding 'ldap_referrals = false' to the domain section of sssd.conf might help to get around the issue?

sbose, thank you very much. Adding ldap_referrals = false to my sssd.conf resolved the issue.

Since using referrals with AD is discouraged and the original reporter confirmed that the issue is resolved, I'm closing this ticket.

resolution: => worksforme
status: new => closed

Sumit convinced me off-list we should take another look before closing. Sorry for the noise.

resolution: worksforme =>
status: closed => reopened

It seems to be the same case as in CentOS.
https://bugs.centos.org/view.php?id=9982.
AD + ldap provider but without disabled referrals.

Fields changed

owner: somebody => pbrezina
status: reopened => new

Fields changed

patch: 0 => 1
status: new => assigned

Fields changed

milestone: NEEDS_TRIAGE => SSSD 1.13.4

master:

sssd-1-13:

resolution: => fixed
status: assigned => closed

Metadata Update from @jayhendren:
- Issue assigned to pbrezina
- Issue set to the milestone: SSSD 1.13.4

2 years ago

Login to comment on this ticket.

Metadata