The nightly test test_simple_replication.py::TestSimpleReplication::test_replica_manage is failing in fedora 33. See PR #647 with the following logs and report:
test_simple_replication.py::TestSimpleReplication::test_replica_manage
self = <ipatests.test_integration.test_simple_replication.TestSimpleReplication object at 0x7f80a27b0520> def test_replica_manage(self): """Test ipa-replica-manage list Ensure that ipa-replica-manage list -v <node> does not print last init status: None last init ended: 1970-01-01 00:00:00+00:00 when the node never had any total update. Test for ticket 7716. """ msg1 = "last init ended: 1970-01-01 00:00:00+00:00" msg2 = "last init status: None" > result = self.master.run_command( ["ipa-replica-manage", "list", "-v", self.replicas[0].hostname]) test_integration/test_simple_replication.py:87: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <ipatests.pytest_ipa.integration.host.Host master.ipa.test (master)> argv = ['ipa-replica-manage', 'list', '-v', 'replica0.ipa.test'], set_env = True stdin_text = None, log_stdout = True, raiseonerr = True, cwd = None, bg = False encoding = 'utf-8', ok_returncode = 0 def run_command(self, argv, set_env=True, stdin_text=None, log_stdout=True, raiseonerr=True, cwd=None, bg=False, encoding='utf-8', ok_returncode=0): """Wrapper around run_command to log stderr on raiseonerr=True :param ok_returncode: return code considered to be correct, you can pass an integer or sequence of integers """ result = super().run_command( argv, set_env=set_env, stdin_text=stdin_text, log_stdout=log_stdout, raiseonerr=False, cwd=cwd, bg=bg, encoding=encoding ) # in FIPS mode SSH may print noise to stderr, remove the string # "FIPS mode initialized" + optional newline. result.stderr_bytes = FIPS_NOISE_RE.sub(b'', result.stderr_bytes) try: result_ok = result.returncode in ok_returncode except TypeError: result_ok = result.returncode == ok_returncode if not result_ok and raiseonerr: result.log.error('stderr: %s', result.stderr_text) > raise subprocess.CalledProcessError( result.returncode, argv, result.stdout_text, result.stderr_text ) E subprocess.CalledProcessError: Command '['ipa-replica-manage', 'list', '-v', 'replica0.ipa.test']' returned non-zero exit status 1. pytest_ipa/integration/host.py:200: CalledProcessError ------------------------------Captured stderr call------------------------------ ipa: ERROR: stderr: Unknown host replica0.ipa.test: Host 'replica0.ipa.test' does not have corresponding DNS A/AAAA record
Another failure: PR648, report
The problem is blocking https://pagure.io/freeipa/issue/8544
Metadata Update from @cheimes: - Custom field blocking adjusted to 8544 - Issue marked as blocking: #8544
I added a DNS lookup before the failing test. Turns out master cannot resolve the replica hostname:
:transport.py:513 RUN ['dig', 'replica0.ipa.test'] :transport.py:557 :transport.py:557 ; <<>> DiG 9.11.27-RedHat-9.11.27-1.fc33 <<>> replica0.ipa.test :transport.py:557 ;; global options: +cmd :transport.py:557 ;; Got answer: :transport.py:557 ;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 27926 :transport.py:557 ;; flags: qr aa rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1 :transport.py:557 :transport.py:557 ;; OPT PSEUDOSECTION: :transport.py:557 ; EDNS: version: 0, flags:; udp: 1232 :transport.py:557 ; COOKIE: fcc42f05843bef4827c08aad601801dfbd1377485560a286 (good) :transport.py:557 ;; QUESTION SECTION: :transport.py:557 ;replica0.ipa.test. IN A :transport.py:557 :transport.py:557 ;; AUTHORITY SECTION: :transport.py:557 ipa.test. 1 IN SOA master.ipa.test. hostmaster.ipa.test. 1612186046 3600 900 1209600 3600 :transport.py:557 :transport.py:557 ;; Query time: 0 msec :transport.py:557 ;; SERVER: 127.0.0.1#53(127.0.0.1) :transport.py:557 ;; WHEN: Mon Feb 01 13:27:59 UTC 2021 :transport.py:557 ;; MSG SIZE rcvd: 128 :transport.py:557 :transport.py:217 Exit code: 0 :transport.py:391 RUN ['ipa-replica-manage', 'list', '-v', 'replica0.ipa.test'] :transport.py:513 RUN ['ipa-replica-manage', 'list', '-v', 'replica0.ipa.test'] :transport.py:557 Unknown host replica0.ipa.test: Host 'replica0.ipa.test' does not have corresponding DNS A/AAAA record :transport.py:217 Exit code: 1
The root cause has been identified. PRCI is creating /etc/hosts file on the master/replica/client machines with a line for each test machine. In fedora33, systemd-resolved is used and "dig@127.0.0.53 client.ipa.test" on the client.ipa.test machine is able to answer (because systemd-resolved is configured by default with ReadEtcHosts=true, it also gets answers from the local /etc/hosts content). As a consequence, ipa-client-install is tricked into believing that an A/AAAA record is already defined for the client machine and does not create the record (see https://github.com/freeipa/freeipa/blob/3a584803da697b80d49590223be632a7451e1b7e/ipaclient/install/client.py#L1564).
On fedora32, the client installer correctly sees that no DNS record exists for the client and adds the record in IPA DNS. That explains the difference between fedora32 and fodora33.
Another failure: PR680, report
Fixed on the latest nightlies: - testing_master_previous: PR 780, Report - testing_master_latest: PR 779, Report - testing_ipa-4-9_latest: PR 771, Report - testing_ipa-4-9_previous: PR 773, Report - testing_ipa-4.6: PR 776, Report
thanks to PR https://github.com/freeipa/freeipa/pull/5474 DNS resolvers management
Metadata Update from @frenaud: - Issue close_status updated to: fixed - Issue status updated to: Closed (was: Open)
Login to comment on this ticket.