#3380 sssd will resolve groups with different domain than default_domain_suffix without domain
Closed: cloned-to-github 3 years ago by pbrezina. Opened 6 years ago by orion.

We have:

domains = nwra.com
full_name_format = %1$s
default_domain_suffix = ad.nwra.com

Group "orion_hbac" exists in the nwra.com domain, but not the ad.nwra.com domain. With a fresh cache:

# getent group orion_hbac
# getent group orion_hbac@nwra.com
orion_hbac:*:8744:orion
# getent group orion_hbac
orion_hbac:*:8744:orion

Seen with sssd-1.15.2-2.fc26.x86_64 and sssd-1.14.0-43.el7_3.14.x86_64


Also seen with sssd-1.14.0-43.el7.x86_64

Please provide full (maybe sanitized) sssd.conf and not just part of it.
It would be also good to attach sssd_nss.log + sssd_$domain.log with high debug_level in nss and domain section of sssd.conf. It would be good to provide logs from fedora 26 with 1.15.2 rather then el7

To be more clear, ad.nwra.com is the trusted AD domain. sssd.conf:

[domain/nwra.com]
cache_credentials = True
krb5_store_password_if_offline = True
ipa_domain = nwra.com
id_provider = ipa
auth_provider = ipa
access_provider = ipa
ldap_tls_cacert = /etc/ipa/ca.crt
chpass_provider = ipa
ipa_server = _srv_
dns_discovery_domain = nwra.com
ipa_automount_location = boulder
debug_level = 5

[sssd]
services = nss, sudo, pam, ssh, autofs
config_file_version = 2
domains = nwra.com
full_name_format = %1$s
default_domain_suffix = ad.nwra.com
debug_level = 5

[nss]
# Default of 60 creates a race with crond
# https://bugzilla.redhat.com/show_bug.cgi?id=1209600
client_idle_timeout = 75
default_shell = /bin/bash
debug_level = 5
override_homedir = /home/%u

[pam]
debug_level = 5

[sudo]

[autofs]

[ssh]

[pac]

[ifp]

I will note the for the subsequent "successful" "getent group orion_hbac" calls, no more sssd log entries are generated and getent appears to be pulling the entry from /var/lib/sss/mc/group:

open("/lib64/libnss_sss.so.2", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\340\23\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=36512, ...}) = 0
mmap(NULL, 2130672, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fb5a3943000
mprotect(0x7fb5a394b000, 2093056, PROT_NONE) = 0
mmap(0x7fb5a3b4a000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x7000) = 0x7fb5a3b4a000
close(3)                                = 0
mprotect(0x7fb5a3b4a000, 4096, PROT_READ) = 0
munmap(0x7fb5a46a9000, 186163)          = 0
open("/var/lib/sss/mc/group", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=6406312, ...}) = 0
mmap(NULL, 6406312, PROT_READ, MAP_SHARED, 3, 0) = 0x7fb5a3326000
fstat(3, {st_mode=S_IFREG|0644, st_size=6406312, ...}) = 0
fstat(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 1), ...}) = 0
write(1, "orion_hbac:*:8744:orion\n", 24orion_hbac:*:8744:orion
) = 24
exit_group(0)                           = ?

And so running:

SSS_NSS_USE_MEMCACHE=no getent group orion_hbac

returns nothing.

Metadata Update from @jhrozek:
- Custom field rhbz adjusted to https://bugzilla.redhat.com/show_bug.cgi?id=1442461

6 years ago

Metadata Update from @jhrozek:
- Custom field rhbz adjusted to https://bugzilla.redhat.com/show_bug.cgi?id=1442461

6 years ago

Thank you very much for sssd.conf file. FYI rhbz#1209600 is already fixed :-) https://pagure.io/SSSD/sssd/issue/2626

It would be also good to attach sssd_nss.log + sssd_$domain.log with high debug_level in nss and domain section of sssd.conf. It would be good to provide logs from fedora 26 with 1.15.2 rather then el7

Also very easy to test with the admin user:

$ id admin
id: admin: no such user
$ id admin@nwra.com
uid=8000(admin) gid=8000(admins) groups=8000(admins)
$ id admin
uid=8000(admin) gid=8000(admins) groups=8000(admins)

Seems like a problem with the memcache cache not being qualified by the domain, much as was the issue with the sss db initially.

id admin

it will try to find user in AD due to default_domain_suffix = ad.nwra.com. It is expected that it failed. Because AD usually have Administrator and not admin

Could you clarify few things?
Is the group orion_hbac stored in IPA or in AD? Is it an external group?

I do not want to say that there is not a bug in sssd But We need a better steps to reproduce it.

id admin

it will try to find user in AD due to default_domain_suffix = ad.nwra.com. It is expected that it failed. Because AD usually have Administrator and not admin

Right, this is expected. What is not expected is it to work after resolving admin@nwra.com. This would appear to be due to the memcache cache storing the unqualified name. At least that's my assumption.

orion_hbac in an IPA group that contains the IPA external group "orion_external" which contains the AD user orion@ad.nwra.com.

It has nothing to do with mem_cache.

If you call id admin sssd will append default_domain_suffix. You "actually" called id admin@ad.nwra.com and such user does not exist.

If you use fully qualified name id admin@nwra.com then sssd will not append default domain suffix because it recognized it as fully qualified name.

The behaviour with admin is expected.

Maybe this will convince you?

$ id admin
id: admin: no such user
$ id admin@nwra.com
uid=8000(admin) gid=8000(admins) groups=8000(admins)
$ id admin
uid=8000(admin) gid=8000(admins) groups=8000(admins)
$ SSS_NSS_USE_MEMCACHE=no id admin
id: admin: no such user

The explanation is simple:

$ id admin
id: admin: no such user

it was not found because admin@ad.nwra.com does not exist

$ id admin@nwra.com
uid=8000(admin) gid=8000(admins) groups=8000(admins)

You do not have enabled use_fully_qualified_names and therefore short version are returned in output. admins vs admins@nwra.com. And short version of name is stored in memory cache; which explains output of following two commands.

BTW the man sssd.conf says.

       default_domain_suffix (string)
           This string will be used as a default domain name for all names
           without a domain name component. The main use case is environments
           where the primary domain is intended for managing host policies and
           all users are located in a trusted domain. The option allows those
           users to log in just with their user name without giving a domain
           name as well.

           Please note that if this option is set all users from the primary
           domain have to use their fully qualified name, e.g.
           user@domain.name, to log in. Setting this option changes default of
           use_fully_qualified_names to True. It is not allowed to use this
           option together with use_fully_qualified_names set to False.

           Default: not set

It looks like bug that use_fully_qualified_names was not enforced by sssd with enabled default_domain_suffix

It might be caused by https://docs.pagure.org/SSSD.sssd/design_pages/shortnames.html. Difficult to say. It was just a wild guess.

Could you try to explicitly set use_fully_qualified_names = true?

You keep explaining the first two results, which I understand and are fine. It's the next two that show the problem:

$ id admin
uid=8000(admin) gid=8000(admins) groups=8000(admins)

This should NOT return this result. It should return "no such user" just like the first time it was run. And the following shows that it is clearly memcache related as you get a different (and correct) result when disabled:

$ SSS_NSS_USE_MEMCACHE=no id admin
id: admin: no such user

The bug is here

id admin@nwra.com
uid=8000(admin) gid=8000(admins) groups=8000(admins)

it should be

id admin@nwra.com
uid=8000(admin@nwra.com) gid=8000(admins@nwra.com) groups=8000(admins@nwra.com)

Well, we're using "full_name_format = %1$s", so I expect the short name to be reported to the user. But I would expect the cache to have cached "admin@nwra.com" not "admin".

Although reporting the fully qualified name for a fully qualified request does seem acceptable/expected as well.

Well, we're using "full_name_format = %1$s", so I expect the short name to be >reported to the user. But I would expect the cache to have cached "admin >@nwra.com" not "admin".

Thank you for pointing it out. I think you found combination of settings which cannot work. SSSD should not even start with such combination.
In another words, default_domain_suffix with full_name_format = %1$s cannot work.

We need to update documentation of the option default_domain_suffix.

I think your goal is to use shortnames in all domains(including trusted)
In that case, I would recommend to look into feature which was implemented in 1.15.x. And remove options default_domain_suffix full_name_format

https://docs.pagure.org/SSSD.sssd/design_pages/shortnames.html

If something is not clear in documentation or there are some corner-cases/bugs. Please report them.

Thank you very much for your patience.

Why shouldn't sssd start with default_domain_suffix and full_name_format = %1$s? The full_name_format is just the output formatter..

Well, we're using "full_name_format = %1$s", so I expect the short name to be >reported to the user. But I would expect the cache to have cached "admin > @nwra.com" not "admin".

Thank you for pointing it out. I think you found combination of settings which cannot work. SSSD should not even start with such combination.
In another words, default_domain_suffix with full_name_format = %1$s cannot work.
We need to update documentation of the option default_domain_suffix.

Possibly, but effort was made to make this work previously - see https://pagure.io/SSSD/sssd/issue/2838 It would be disappointing so see it given up on now, although I'll check out the other docs. We're mostly EL7 though so 1.15 may be a ways out for us.

Why shouldn't sssd start with default_domain_suffix and full_name_format = %1$s? The full_name_format is just the output formatter..

The phrasing was probably too strict.
But it won't work reliably with mixing queries in shortnames and fully qualified names. Especially in case of conflicting names. Memory cache cannot detect that: admin, admin@exmapl.com and admin@test.example.com are two different users because all of them will have the same output format. But the same problem will be with https://docs.pagure.org/SSSD.sssd/design_pages/shortnames.html

We can detect conflicts in nss responder because name are stored fully qualified there. But we cannot detect it in memory cache.

Possibly, but effort was made to make this work previously - see https://pagure.io/SSSD/sssd/issue/2838 It would be disappointing so see it given up on now, although I'll check out the other docs.

That ticket says it is fixed in 1.14.0 but you mentioned it does not work with sssd-1.14.0-43.el7.x86_64.
Have it ever work correctly for you? Or just some of latest minor updated in el7 broke it?

We're mostly EL7 though so 1.15 may be a ways out for us.

There were done huge changes between 1.14 and 1.15 due to https://docs.pagure.org/SSSD.sssd/design_pages/shortnames.html and few other features. So bug in 1.15 would be very likely different thaen in 1.14

Possibly, but effort was made to make this work previously - see https://pagure.io/SSSD/sssd/issue/2838 It would be disappointing so see it given up on now, although I'll check out the other docs.

That ticket says it is fixed in 1.14.0 but you mentioned it does not work with sssd-1.14.0-43.el7.x86_64.
Have it ever work correctly for you? Or just some of latest minor updated in el7 broke it?

They are different bugs. The first was caused by the use of shortnames in the sssd sysdb database, and that issue is resolved. This one appears to be a similar type of problem but with the memcache rather than sysdb.

I'll summarize the discussion we had with the other SSSD developers yesterday during a phone call. If I misremembered something, hopefully the others will correct me.

Yes, this is an issue with the memory cache. but unfortunately it is not an easy one to solve. Currently, the entries in the memory cache are keyed using the output format. If the output is set to shortname only, the memory cache will contain short names..and because the memory cache is evaluated not in the deamon, but in the application calling getpw* (getent in this case), not much logic is possible.

What might be easier for you than this hack with full_name_format (which I agree is/was the only way to go) is to upgrade to RHEL-7.4 when it's out and instead of changing the full_name_format, change the configuration to contain use_fully_qualified_names=False for the AD domains, but keep qualified names for the IPA domain in output (since the input needs to be qualified anyway).

Longer-term, we need to change the memory cache to be keyed with what the the input was. Since we need to do more work in the memory cache, like support for by-SID lookups, I would like to move this ticket to the Future Releases milestone. Hopefully we'd get to the memory cache improvements in the next update (which would be in 7.5), but we haven't even started planning that one.

Metadata Update from @jhrozek:
- Issue set to the milestone: SSSD Future releases (no date set yet)

6 years ago

Metadata Update from @jhrozek:
- Issue set to the milestone: SSSD 1.16.0 (was: SSSD Future releases (no date set yet))

6 years ago

Metadata Update from @jhrozek:
- Issue priority set to: major

6 years ago

Since we are required to release a new upstream tarball no later than Friday Oct-20, I'm moving tickets that will not be closed by that date to the next milestone, 1.16.1

Metadata Update from @jhrozek:
- Issue set to the milestone: SSSD 1.16.1 (was: SSSD 1.16.0)

6 years ago

Metadata Update from @jhrozek:
- Issue tagged with: postpone-to-2-0

6 years ago

Metadata Update from @jhrozek:
- Issue untagged with: postpone-to-2-0
- Issue set to the milestone: SSSD 2.0 (was: SSSD 1.16.1)

6 years ago

Metadata Update from @jhrozek:
- Issue set to the milestone: SSSD 2.1 (was: SSSD 2.0)

5 years ago

Metadata Update from @jhrozek:
- Issue set to the milestone: SSSD 2.2 (was: SSSD 2.1)

5 years ago

Metadata Update from @jhrozek:
- Issue set to the milestone: SSSD 2.3 (was: SSSD 2.2)

4 years ago

Metadata Update from @pbrezina:
- Issue tagged with: bugzilla

4 years ago

SSSD is moving from Pagure to Github. This means that new issues and pull requests
will be accepted only in SSSD's github repository.

This issue has been cloned to Github and is available here:
- https://github.com/SSSD/sssd/issues/4408

If you want to receive further updates on the issue, please navigate to the github issue
and click on subscribe button.

Thank you for understanding. We apologize for all inconvenience.

Metadata Update from @pbrezina:
- Issue close_status updated to: cloned-to-github
- Issue status updated to: Closed (was: Open)

3 years ago

Login to comment on this ticket.

Metadata
Attachments 1
Attached 6 years ago View Comment