#8688 Multiple nightly test failures in test_integration/test_caless.py::TestReplicaInstall
Closed: insufficientinfo 3 years ago by frenaud. Opened 3 years ago by mpolovka.

test_caless.py::TestReplicaInstall::()::test_no_http_password       
test_caless.py::TestReplicaInstall::()::test_no_ds_password
test_caless.py::TestReplicaInstall::()::test_certs_with_no_password         
test_caless.py::TestReplicaInstall::()::test_certs_with_no_password_interactive

failed in testing_ipa-4.6 PR685 with

    @replica_install_teardown
    def test_certs_with_no_password_interactive(self):
        # related to https://pagure.io/freeipa/issue/7274

        self.create_pkcs12('ca1/replica', filename='http.p12',
                           password='')
        self.create_pkcs12('ca1/replica', filename='dirsrv.p12',
                           password='')
        self.prepare_cacert('ca1')
        stdin_text = '\n\nyes'

        result = self.prepare_replica(http_pkcs12='http.p12',
                                      dirsrv_pkcs12='dirsrv.p12',
                                      http_pin=None, dirsrv_pin=None,
                                      unattended=False, stdin_text=stdin_text)
>       assert result.returncode == 0
E       AssertionError: assert 2 == 0
E        +  where 2 = <pytest_multihost.transport.SSHCommand object at 0x7fd73d6eceb8>.returncode

test_integration/test_caless.py:1186: AssertionError

During handling of the above exception, another exception occurred:

args = (<ipatests.test_integration.test_caless.TestReplicaInstall object at 0x7fd73d6ed4e0>,)
replica = <ipatests.pytest_ipa.integration.host.Host replica0.ipa.test (replica)>
master = <ipatests.pytest_ipa.integration.host.Host master.ipa.test (master)>

    def wrapped(*args):
        try:
            func(*args)
        finally:
            # Uninstall replica
            replica = args[0].replicas[0]
            master = args[0].master
>           tasks.kinit_admin(master)

test_integration/test_caless.py:108: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
pytest_ipa/integration/tasks.py:910: in kinit_admin
    stdin_text=host.config.admin_password)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <ipatests.pytest_ipa.integration.host.Host master.ipa.test (master)>
argv = ['kinit', 'admin'], set_env = True, stdin_text = 'Secret.123'
log_stdout = True, raiseonerr = True, cwd = None, ok_returncode = 0

    def run_command(self, argv, set_env=True, stdin_text=None,
                    log_stdout=True, raiseonerr=True,
                    cwd=None, ok_returncode=0):
        """Wrapper around run_command to log stderr on raiseonerr=True

            :param ok_returncode: return code considered to be correct,
                                  you can pass an integer or sequence of integers
            """
        result = super(Host, self).run_command(
            argv, set_env=set_env, stdin_text=stdin_text,
            log_stdout=log_stdout, raiseonerr=False, cwd=cwd
        )
        try:
            result_ok = result.returncode in ok_returncode
        except TypeError:
            result_ok = result.returncode == ok_returncode
        if not result_ok and raiseonerr:
            result.log.error('stderr: %s', result.stderr_text)
            raise subprocess.CalledProcessError(
                result.returncode, argv,
>               result.stdout_text
            )
E           subprocess.CalledProcessError: Command '['kinit', 'admin']' returned non-zero exit status 1.

logs also show

[ipatests.pytest_ipa.integration.host.Host.master.cmd502] RUN ['ipa', 'dnszone-add', '122.168.192.in-addr.arpa.']
[ipatests.pytest_ipa.integration.host.Host.master.cmd502] ipa: ERROR: Major (851968): Unspecified GSS failure.  Minor code may provide more information, Minor (2529638972): KDC returned error string: PROCESS_TGS
[ipatests.pytest_ipa.integration.host.Host.master.cmd502] Exit code: 1
ipa: WARNING: ipa: ERROR: Major (851968): Unspecified GSS failure.  Minor code may provide more information, Minor (2529638972): KDC returned error string: PROCESS_TGS

[ipatests.pytest_ipa.integration.tasks] ipa: ERROR: Major (851968): Unspecified GSS failure.  Minor code may provide more information, Minor (2529638972): KDC returned error string: PROCESS_TGS

Full report


The issue seems to be caused by a slapd coredump: in the journal:

an 31 16:28:18 master.ipa.test ns-slapd[16811]: [31/Jan/2021:16:28:18.133048510 +0000] - ERR - ipa-topology-plugin - ipa_topo_agmt_del: cn=meToreplica0.ipa.test
Jan 31 16:28:18 master.ipa.test ns-slapd[16811]: [31/Jan/2021:16:28:18.262517880 +0000] - ERR - ipa-topology-plugin - ipa_topo_util_modify: failed to modify entry (cn=cn=replica0.ipa.test-to-master.ipa.test,cn=domain,cn=topology,cn=ipa,cn=etc,dc=ipa,dc=test): error 32
Jan 31 16:28:18 master.ipa.test kernel: ns-slapd[16827]: segfault at 565200000003 ip 00007f7228ddbf31 sp 00007f720ad752d8 error 4 in libc-2.26.so[7f7228c86000+1ad000]
Jan 31 16:28:18 master.ipa.test kernel: Code: 2e 0f 1f 84 00 00 00 00 00 31 c0 c5 f8 77 c3 66 2e 0f 1f 84 00 00 00 00 00 89 f9 48 89 fa c5 f9 ef c0 83 e1 3f 83 f9 20 77 1f <c5> fd 74 0f c5 fd d7 c1 85 c0 0f 85 df 00 00 00 48 83 c7 20 83 e1 
Jan 31 16:28:18 master.ipa.test audit[16811]: ANOM_ABEND auid=4294967295 uid=389 gid=389 ses=4294967295 pid=16811 comm="ns-slapd" exe="/usr/sbin/ns-slapd" sig=11 res=1

After this crash, the kdc can't issue tickets any more because it can't connect to the LDAP server, see kdc log:

Jan 31 16:28:20 master.ipa.test krb5kdc[16839](info): TGS_REQ (8 etypes {18 17 20 19 16 23 25 26}) 192.168.122.151: PROCESS_TGS: authtime 0,  <unknown client> for HTTP/master.ipa.test@IPA.TEST, Server error

The issue happened only once + we don't have enough data to analyze (no coredump), closing as insufficientinfo.

Metadata Update from @frenaud:
- Issue close_status updated to: insufficientinfo
- Issue status updated to: Closed (was: Open)

3 years ago

Login to comment on this ticket.

Metadata