The nightly tests are failing on rawhide during ipa-server-install --uninstall. See PR #416, with for instance automember (report, logs):
ipa-server-install --uninstall
automember
RUN ['ipa-server-install', '--uninstall', '-U', '--ignore-topology-disconnect', '--ignore-last-of-role'] RUN ['ipa-server-install', '--uninstall', '-U', '--ignore-topology-disconnect', '--ignore-last-of-role'] Updating DNS system records unable to resolve host name master.ipa.test. to IP address, ipa-ca DNS record will be incomplete Forcing removal of master.ipa.test Ignoring topology connectivity errors. ------------------------------------ Deleted IPA server "master.ipa.test" ------------------------------------ Shutting down all IPA services Unconfiguring CA org.freedesktop.DBus.Error.NoReply: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken. The ipa-server-install command failed. See /var/log/ipaserver-uninstall.log for more information Exit code: 1
Closing as a duplicate of https://pagure.io/freeipa/issue/8533
Metadata Update from @rcritten: - Issue close_status updated to: duplicate - Issue status updated to: Closed (was: Open)
Metadata Update from @frenaud: - Issue status updated to: Open (was: Closed)
The issue is still happening, see for instance PR #496 with the test test_caless_TestReplicaCALessToCAFull (report, logs):
test_caless_TestReplicaCALessToCAFull
[ipatests.pytest_ipa.integration.host.Host.replica0.cmd106] RUN ['ipa-server-install', '--uninstall', '-U', '--ignore-topology-disconnect', '--ignore-last-of-role'] [ipatests.pytest_ipa.integration.host.Host.replica0.cmd106] Updating DNS system records [ipatests.pytest_ipa.integration.host.Host.replica0.cmd106] Forcing removal of replica0.ipa.test [ipatests.pytest_ipa.integration.host.Host.replica0.cmd106] Ignoring topology connectivity errors. [ipatests.pytest_ipa.integration.host.Host.replica0.cmd106] Failed to cleanup replica0.ipa.test DNS entries: no matching entry found [ipatests.pytest_ipa.integration.host.Host.replica0.cmd106] You may need to manually remove them from the tree [ipatests.pytest_ipa.integration.host.Host.replica0.cmd106] -------------------------------------- [ipatests.pytest_ipa.integration.host.Host.replica0.cmd106] Deleted IPA server "replica0.ipa.test" [ipatests.pytest_ipa.integration.host.Host.replica0.cmd106] -------------------------------------- [ipatests.pytest_ipa.integration.host.Host.replica0.cmd106] Shutting down all IPA services [ipatests.pytest_ipa.integration.host.Host.replica0.cmd106] Unconfiguring CA [ipatests.pytest_ipa.integration.host.Host.replica0.cmd106] org.freedesktop.DBus.Error.NoReply: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken. [ipatests.pytest_ipa.integration.host.Host.replica0.cmd106] The ipa-server-install command failed. See /var/log/ipaserver-uninstall.log for more information [ipatests.pytest_ipa.integration.host.Host.replica0.cmd106] Exit code: 1
Similar issue observed in PR 526 with test test_integration/test_advise.py::TestAdvice::test_advice_config_server_for_smart_card_auth : logs and test_cert : logs
Also on master branch with selinux enabled: PR #536
In #8586, I have a duplicate of this bug. I have a VM after the failed uninstall, its currently suspended (not powered off). If anyone wants to use it to debug, let me know.
The replica install fails here while trying to start tracking a cert:
http://freeipa-org-pr-ci.s3-website.eu-central-1.amazonaws.com/jobs/73433c76-243f-11eb-afef-fa163e46012f/test_integration-test_cert.py-TestCertmongerInterruption-install/
What's strange is I see almost no certmonger output at all in the log. IPA sets it to -d2 which should be somewhat chatty. This leads me to believe that the problem is in the installer trying to make the D-Bus request to certmonger and that timing out. It doesn't seem to have gotten to certmonger at all.
Similarly the related test uninstall (separate directory) is failing to start certmonger because it cannot connect to D-Bus. The uninstaller verifies that the dbus service is active immediately prior to this.
My theory is that dbus has gone out to lunch.
Similar issue observed in [testing_master_pki] Nightly PR #535 report
This issue is happening very regularly and needs to be investigated.
The problem is now blocking https://pagure.io/freeipa/issue/8544 . The PR depends on a new DS feature that is only available in most recent update in Fedora 33.
Metadata Update from @cheimes: - Custom field blocking adjusted to 8544 - Issue marked as blocking: #8544
So it reproducibly fails? I've never been able to reproduce it outside of PR-CI myself.
It reproducible fails on PR-CI...
Failure also observed: PR680, test _cert_fix and also test_commands and also test_forced_client_enrolment and also test_pwpolicy
master:
ipa-4-9:
Metadata Update from @abbra: - Issue close_status updated to: fixed - Issue status updated to: Closed (was: Open)
Similar failure observed in PR 738 Logs
Issue still present in [testing_ipa-4.9_latest] Nightly PR #741: test_ipahealthcheck, logs
test_ipahealthcheck
Issue still present in [testing_master_pki] Nightly PR #762 test_ipahealthcheck, logs
Still present in [testing_master_previous] Nightly PR #765 , report - test_ipahealthcheck
Metadata Update from @rcritten: - Custom field affects_doc adjusted to on - Custom field knownissue adjusted to on - Issue status updated to: Open (was: Closed)
PR https://github.com/freeipa/freeipa/pull/5573
Metadata Update from @rcritten: - Custom field rhbz adjusted to https://bugzilla.redhat.com/show_bug.cgi?id=1930038
Issue linked to Bugzilla: Bug 1930038
Metadata Update from @rcritten: - Issue assigned to rcritten
Metadata Update from @frenaud: - Issue close_status updated to: fixed - Issue status updated to: Closed (was: Open)
Issue still present in [testing_master_previous] Nightly PR #832, report
Issue resurfaced in test_integration/test_ipa_cert_fix.py::TestIpaCertFix::test_missing_csr, [testing_ipa-4.9_latest_selinux] Nightly PR #965 , report
test_integration/test_ipa_cert_fix.py::TestIpaCertFix::test_missing_csr
It's the same symptom but a different broken test. Tests that move around time are problematic for certmonger since it is going to aggressively try to renew certificates. It also needs to be running during uninstallation so that the certificates can be untracked. In this case it is trying to do that but IPA isn't running so its getting blocked up enough that the DBus timeout is invoked.
One fix, since we don't care at all about the certs, would be to rm -f paths.CERTMONGER_REQUESTS_DIR/* while ipa and certmonger are shut down. Then there is nothing to do.
https://github.com/freeipa/freeipa/pull/6177
Login to comment on this ticket.