#8866 [Tracker] Nightly failure in test_installation_TestInstallMaster: systemd-resolve coredump
Opened 3 years ago by frenaud. Modified 2 years ago

The nightly test testing-fedora/test_installation_TestInstallMaster failed in PR #924 [testing_master_testing] on fedora 34.

Available logs and report:

self = <ipatests.test_integration.test_installation.TestInstallMaster object at 0x7ff9ecb71430>

    def test_ipaca_no_redirect(self):
        """Test that ipa-ca.$DOMAIN does not redirect

           ipa-ca is a valid name for an IPA server. It should not
           require a redirect.

           CRL generation does not need to be enabled for this test.
           We aren't exactly testing that a CRL can be retrieved, just
           that the redirect doesn't happen.
        """

        def run_request(url, expected_stdout=None, expected_stderr=None):
            result = self.master.run_command(['curl', '-s', '-v', url])
            if expected_stdout:
                assert expected_stdout in result.stdout_text
            if expected_stderr:
                assert expected_stderr in result.stderr_text

        # CRL publishing on start-up is disabled so drop a file there
        crlfile = os.path.join(paths.PKI_CA_PUBLISH_DIR, 'MasterCRL.bin')
        self.master.put_file_contents(crlfile, 'secret')

        hosts = (
            f'{IPA_CA_RECORD}.{self.master.domain.name}',
            self.master.hostname,
        )

        # Positive tests. Both hosts can serve these.
        urls = (
            'http://{host}/ipa/crl/MasterCRL.bin',
            'http://{host}/ca/ocsp',
            'https://{host}/ca/admin/ca/getCertChain',
            'https://{host}/acme/',
        )
        for url in urls:
            for host in hosts:
>               run_request(
                    url.format(host=host),
                    expected_stderr='HTTP/1.1 200'
                )

test_integration/test_installation.py:1110: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
test_integration/test_installation.py:1086: in run_request
    result = self.master.run_command(['curl', '-s', '-v', url])
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <ipatests.pytest_ipa.integration.host.Host master.ipa.test (master)>
argv = ['curl', '-s', '-v', 'http://ipa-ca.ipa.test/ipa/crl/MasterCRL.bin']
set_env = True, stdin_text = None, log_stdout = True, raiseonerr = True
cwd = None, bg = False, encoding = 'utf-8', ok_returncode = 0

    def run_command(self, argv, set_env=True, stdin_text=None,
                    log_stdout=True, raiseonerr=True,
                    cwd=None, bg=False, encoding='utf-8', ok_returncode=0):
        """Wrapper around run_command to log stderr on raiseonerr=True

        :param ok_returncode: return code considered to be correct,
                              you can pass an integer or sequence of integers
        """
        result = super().run_command(
            argv, set_env=set_env, stdin_text=stdin_text,
            log_stdout=log_stdout, raiseonerr=False, cwd=cwd, bg=bg,
            encoding=encoding
        )
        # in FIPS mode SSH may print noise to stderr, remove the string
        # "FIPS mode initialized" + optional newline.
        result.stderr_bytes = FIPS_NOISE_RE.sub(b'', result.stderr_bytes)
        try:
            result_ok = result.returncode in ok_returncode
        except TypeError:
            result_ok = result.returncode == ok_returncode
        if not result_ok and raiseonerr:
            result.log.error('stderr: %s', result.stderr_text)
>           raise subprocess.CalledProcessError(
                result.returncode, argv,
                result.stdout_text, result.stderr_text
            )
E           subprocess.CalledProcessError: Command '['curl', '-s', '-v', 'http://ipa-ca.ipa.test/ipa/crl/MasterCRL.bin']' returned non-zero exit status 6.

pytest_ipa/integration/host.py:202: CalledProcessError
 ------------------------------Captured stderr call------------------------------ 
ipa: ERROR: stderr: * Could not resolve host: ipa-ca.ipa.test
* Closing connection 0

The test fails because the hostname ipa-ca.ipa.test cannot be resolved to an IP address.

The journal shows that systemd-resolve coredumped:

May 23 03:04:57 master.ipa.test systemd[1]: Started Process Core Dump (PID 40022/UID 0).
May 23 03:04:57 master.ipa.test audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-coredump@0-40022-0 comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
May 23 03:04:57 master.ipa.test systemd-coredump[40023]: Process 33792 (systemd-resolve) of user 193 dumped core.

                                                         Stack trace of thread 33792:
                                                         #0  0x00007f636947e2a2 raise (libc.so.6 + 0x3d2a2)
                                                         #1  0x00007f63694678a4 abort (libc.so.6 + 0x268a4)
                                                         #2  0x00007f6369b7e732 log_assert_failed.cold (libsystemd-shared-248.so + 0x78732)
                                                         #3  0x00005600c3c941b5 on_stream_io (systemd-resolved + 0x261b5)
                                                         #4  0x00007f6369d203e4 source_dispatch.lto_priv.0 (libsystemd-shared-248.so + 0x21a3e4)
                                                         #5  0x00007f6369d22d55 sd_event_dispatch (libsystemd-shared-248.so + 0x21cd55)
                                                         #6  0x00007f6369d237e8 sd_event_run (libsystemd-shared-248.so + 0x21d7e8)
                                                         #7  0x00007f6369d239af sd_event_loop (libsystemd-shared-248.so + 0x21d9af)
                                                         #8  0x00005600c3cbb904 run (systemd-resolved + 0x4d904)
                                                         #9  0x00005600c3c7e0ca main (systemd-resolved + 0x100ca)
                                                         #10 0x00007f6369468b75 __libc_start_main (libc.so.6 + 0x27b75)
                                                         #11 0x00005600c3c7e16e _start (systemd-resolved + 0x1016e)

Companion issue opened against fedora34 systemd: https://bugzilla.redhat.com/show_bug.cgi?id=1966543
Installed version of systemd: systemd-248.3-1.fc34.x86_64


The issue also happened in PR #996 [testing_master_latest].

Also seen in PR #1316 with systemd-248.6-1.fc34.x86_64

Also seen in PR #1813 with systemd-resolved-250.7-1.fc36.x86_64 (Logs)

Log in to comment on this ticket.

Metadata