#8189 NIghtly test failure in test_integration/test_nfs.py::TestIpaClientAutomountFileRestore::test_nsswitch_backup_restore_sssd
Opened 5 months ago by frenaud. Modified a month ago

The nightly test [testing_master_rawhide] Nightly PR failed in test_integration/test_nfs.py::TestIpaClientAutomountFileRestore::test_nsswitch_backup_restore_sssd in PR #132. Logs available at the following location.

The test is comparing the content of /etc/nsswitch.conf before backup and after restore. Logs:

self = <ipatests.test_integration.test_nfs.TestIpaClientAutomountFileRestore object at 0x7fb4fdd106d0>

    def test_nsswitch_backup_restore_sssd(self):
>       self.nsswitch_backup_restore()

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <ipatests.test_integration.test_nfs.TestIpaClientAutomountFileRestore object at 0x7fb4fdd106d0>
no_sssd = False

    def nsswitch_backup_restore(

        # In order to get a more pure sum, one that ignores the Generated
        # header and any white space we have to do a bit of work...
        sha256nsswitch_cmd = \
            'egrep -v "Generated|^$" /etc/nsswitch.conf | sed "s/\\s//g" ' \
            '| sort | sha256sum'

        cmd = self.clients[0].run_command(sha256nsswitch_cmd)
        orig_sha256 = cmd.stdout_text

        grep_automount_command = \
            "grep automount /etc/nsswitch.conf | cut -d: -f2"

        tasks.install_client(self.master, self.clients[0])
        cmd = self.clients[0].run_command(grep_automount_command)
        after_ipa_client_install = cmd.stdout_text.split()

        if no_sssd:
            ipa_client_automount_command = [
                "ipa-client-automount", "--no-sssd", "-U"
            ipa_client_automount_command = [
                "ipa-client-automount", "-U"
        cmd = self.clients[0].run_command(grep_automount_command)
        after_ipa_client_automount = cmd.stdout_text.split()
        if no_sssd:
            assert after_ipa_client_automount == ['files', 'ldap']
            assert after_ipa_client_automount == ['sss', 'files']

        cmd = self.clients[0].run_command(grep_automount_command)
        assert cmd.stdout_text.split() == after_ipa_client_automount

            "ipa-client-automount", "--uninstall", "-U"

        cmd = self.clients[0].run_command(grep_automount_command)
        assert cmd.stdout_text.split() == after_ipa_client_install

        cmd = self.clients[0].run_command(sha256nsswitch_cmd)
>       assert cmd.stdout_text == orig_sha256
E       AssertionError: assert '8dfaa614f35a...9f7a5d6a  -\n' == 'e83aaf5026f5...5907c10e  -\n'
E         - 8dfaa614f35a8ec334ca4ec5b115d49a1abdba6df78f3afd812fa69c9f7a5d6a  -
E         + e83aaf5026f57684ae30f5ad4da67e5ebeae520c91b935efbd47745a5907c10e  -

test_integration/test_nfs.py:335: AssertionError

See also issue #8063 with the same logs on fedora-29 which got closed.

Metadata Update from @fcami:
- Issue assigned to fcami

5 months ago

Taking for investigation

Actual diff between the original rawhide system and after ipa-client-install + ipa-client-install --uninstall:

diff --git a/nsswitch.conf b/nsswitch.conf
index 6b921d9..a031af8 100644
--- a/nsswitch.conf
+++ b/nsswitch.conf
@@ -1,3 +1,32 @@
+# Generated by authselect on Fri Feb  7 20:00:35 2020
+# Do not modify this file manually.
+# If you want to make changes to nsswitch.conf please modify
+# /etc/authselect/user-nsswitch.conf and run 'authselect apply-changes'.
+# Note that your changes may not be applied as they may be
+# overwritten by selected profile. Maps set in the authselect
+# profile takes always precedence and overwrites the same maps
+# set in the user file. Only maps that are not set by the profile
+# are applied from the user file.
+# For example, if the profile sets:
+#     passwd: sss files
+# and /etc/authselect/user-nsswitch.conf contains:
+#     passwd: files
+#     hosts: files dns
+# the resulting generated nsswitch.conf will be:
+#     passwd: sss files # from profile
+#     hosts: files dns  # from user file
+passwd:     sss files systemd
+group:      sss files systemd
+netgroup:   sss files
+automount:  sss files
+services:   sss files
+# Included from /etc/authselect/user-nsswitch.conf
 # /etc/nsswitch.conf
@@ -36,9 +65,7 @@
 # shadow:    db files
 # group:     db files

-passwd:      sss files systemd
 shadow:     files sss
-group:       sss files systemd

 hosts:      files dns myhostname

@@ -49,11 +76,8 @@ netmasks:   files
 networks:   files
 protocols:  files
 rpc:        files
-services:   files sss

-netgroup:   sss

 publickey:  files

-automount:  files sss
 aliases:    files

At ipa-client-install time we get:

DEBUG    ipatests.pytest_ipa.integration.host.Host.master.cmd13:transport.py:557 WARNING: The configuration pre-client installation is not managed by authselect and cannot be backed up. Uninstallation may not be able to revert to the original state.

This makes sense as the default authselect configuration is exactly this: no profile.

# authselect current
No existing configuration detected.

Enrolling the client:

# ipa-client-install --mkhomedir
# authselect current
Profile ID: sssd
Enabled features:
- with-mkhomedir
- with-sudo


# ipa-client-install --uninstall -U
# authselect current
Profile ID: sssd
Enabled features: None

I have verified that user-nsswitch.conf is identical post-uninstall as it was originally.
The diff between the original nsswitch.conf and the post-uninstall nsswitch.conf is reduced to:

--- initial 2020-02-07 23:47:48.685241496 +0100
+++ post    2020-02-07 23:48:01.604320748 +0100
@@ -2 +2 @@
-automount:  files sss
+automount:  sss files
@@ -5 +5 @@
-group:       sss files systemd
+group:      sss files systemd
@@ -7 +7 @@
-netgroup:   sss
+netgroup:   sss files
@@ -10 +10 @@
-passwd:      sss files systemd
+passwd:     sss files systemd
@@ -14 +14 @@
-services:   files sss
+services:   sss files

which makes sense as authselect ends up with the sssd profile.
The actual fix is to restore the authselect configuration from backup:

# authselect backup-list
2020-02-07-22-33-43.M8fOg6 (created at Fri 07 Feb 2020 11:33:43 PM CET)
2020-02-07-22-38-28.Uukas2 (created at Fri 07 Feb 2020 11:38:28 PM CET)

The first entry is the backup done right before ipa-client-install - the nsswitch.conf is identical to the default. But restoring that backup fails:

# authselect backup-restore 2020-02-07-22-33-43.M8fOg6
[error] Unable to restore [2020-02-07-22-33-43.M8fOg6] [2]: No such file or directory
Unable to restore backup [2020-02-07-22-33-43.M8fOg6] [2]: No such file or directory

Using strace, we see it successively choke on:

openat(AT_FDCWD, "/var/lib/authselect/backups/2020-02-07-22-33-43.M8fOg6/dconf-db", O_RDONLY) = -1 ENOENT (No such file or directory)


openat(AT_FDCWD, "/var/lib/authselect/backups/2020-02-07-22-33-43.M8fOg6/dconf-locks", O_RDONLY) = -1 ENOENT (No such file or directory)

Simply creating empty files:

# touch /var/lib/authselect/backups/2020-02-07-22-33-43.M8fOg6/dconf-db
# touch /var/lib/authselect/backups/2020-02-07-22-33-43.M8fOg6/dconf-locks

makes it work:

# authselect backup-restore 2020-02-07-22-33-43.M8fOg6

and the backuped config is now identical to the original:

config]# md5sum nsswitch.conf /etc/nsswitch.conf
5cc1a8341dbebe35bab636912a0e4b47  nsswitch.conf
5cc1a8341dbebe35bab636912a0e4b47  /etc/nsswitch.conf

- we need to enhance ipaplatform/redhat/authconfig.py to save the ID of the backup after ipa-client-install into ipabackup
- and to use use authselect backup-restore mechanism at uninstall time, using the above ID
- authselect must be fixed before that as the above smells like a bug.

authselect PR: https://github.com/pbrezina/authselect/pull/202

Please, find test builds for f32 [1] or rawhide [2]

[1] https://koji.fedoraproject.org/koji/taskinfo?taskID=41467298
[2] https://koji.fedoraproject.org/koji/taskinfo?taskID=41467290

we need to enhance ipaplatform/redhat/authconfig.py to save the ID of the backup after ipa-client-install into ipabackup

You can also call authselect select sssd --force --backup=NAME to provide a custom backup name.

Metadata Update from @fcami:
- Custom field rhbz adjusted to https://bugzilla.redhat.com/show_bug.cgi?id=1810179

4 months ago

authselect is now considering adding a minimal profile so there are two possibilities: either name the backup or go back to the upcoming minimal profile.

Minimal profile is not the default profile so it would be better to use backup mechanism a return to whatever was there before IPA installation.

Login to comment on this ticket.