#7545 TestCASpecificRUVs.test_replica_uninstall_deletes_ruvs start failing with assertion error
Closed: fixed 2 years ago Opened 2 years ago by gkaihoro.

TestCASpecificRUVs.test_replica_uninstall_deletes_ruvs start failing with assertion error :

Error Message

AssertionError: Replica RUVs were not clean during replica uninstallation assert 'vm-171-238....q.redhat.com' not in 'Replica Upda...com:389: 6\n' 'vm-171-238.abc.idm....eng.brq.redhat.com' is contained here: Replica Update Vectors: vm-171-229.abc.idm.lab.eng.brq.redhat.com:389: 4 vm-069.abc.idm.lab.eng.brq.redhat.com:389: 3 vm-171-238.abc.idm.lab.eng.brq.redhat.com:389: 7 Certificate Server Replica Update Vectors: vm-171-229.abc.idm.lab.eng.brq.redhat.com:389: 6

Stacktrace

self = <ipatests.test_integration.test_topology.TestCASpecificRUVs object at 0x7fd880058668>

    def test_replica_uninstall_deletes_ruvs(self):
        """
            http://www.freeipa.org/page/V4/Manage_replication_topology_4_4/Test_Plan
            #Test_case:_.2A-ruv_subcommands_of_ipa-replica-manage_are_extended
            _to_handle_CA-specific_RUVs
            """
        master = self.master
        replica = self.replicas[1]
        res1 = master.run_command(['ipa-replica-manage', 'list-ruv', '-p',
                                  master.config.dirman_password]).stdout_text
        assert(res1.count(replica.hostname) == 2), (
            "Did not find proper number of replica hostname (%s) occurrencies"
            " in the command output: %s" % (replica.hostname, res1))

        master.run_command(['ipa-replica-manage', 'del', replica.hostname,
                            '-p', master.config.dirman_password])
        tasks.uninstall_master(replica)
        res2 = master.run_command(['ipa-replica-manage', 'list-ruv', '-p',
                                  master.config.dirman_password]).stdout_text
>       assert(replica.hostname not in res2), (
            "Replica RUVs were not clean during replica uninstallation")
E       AssertionError: Replica RUVs were not clean during replica uninstallation
E       assert 'vm-171-238....q.redhat.com' not in 'Replica Upda...com:389: 6\n'
E         'vm-171-238.abc.idm....eng.brq.redhat.com' is contained here:
E           Replica Update Vectors:
E               vm-171-229.abc.idm.lab.eng.brq.redhat.com:389: 4
E               vm-069.abc.idm.lab.eng.brq.redhat.com:389: 3
E               vm-171-238.abc.idm.lab.eng.brq.redhat.com:389: 7
E           Certificate Server Replica Update Vectors:
E               vm-171-229.abc.idm.lab.eng.brq.redhat.com:389: 6

/usr/lib/python3.6/site-packages/ipatests/test_integration/test_topology.py:248: AssertionError

Test start failing potentially after commit 8a8b641


Metadata Update from @gkaihoro:
- Issue tagged with: test-failure, tests

2 years ago

Metadata Update from @fbarreto:
- Issue assigned to fbarreto

2 years ago

Metadata Update from @slaykovsky:
- Issue priority set to: normal
- Issue set to the milestone: FreeIPA 4.6.4

2 years ago

test_topology.py is green on PR CI (check nightly test PR https://github.com/freeipa/freeipa/pull/1921) and here https://fedorapeople.org/groups/freeipa/prci/jobs/ce459b9c-579f-11e8-93b8-fa163e15c266/.

I'm still uploading the new vagrant box (updated f27). Once it's done I will rerun the test with it.

Metadata Update from @rcritten:
- Issue set to the milestone: FreeIPA 4.6.5 (was: FreeIPA 4.6.4)

2 years ago

Failure still happens randomly, for instance see test output from 2018-10-30

The issue happens sometimes, when the master is too slow to clean the ruv. For reference, the test scenario is:
1. ipa-replica-manage del $replica
2. (on replica) ipa-server-install --uninstall -U
3. ipa-replica-manage list-ruv to check that the replica is not present any more.

ipa-replica-manage del is removing the host entry, which in turns triggers the topology plugin. The plugin creates clean-ruv tasks (one for the domain, one for o=ipaca if CA is installed on the replica).
389-ds tasks are asynchronous, meaning that ipa-replica-manage del command may return before the task has finished.
As the test does not check the task status, it may also call ipa-replica-manage list-ruv before the clean-ruv task has finished.
We should update the test so that it checks if the task has finished, waiting for up to a configured timeout, and consider that the test failed if the task did not finish or if list-ruv still displays the replica after the task has finished.

Metadata Update from @frenaud:
- Custom field on_review adjusted to https://github.com/freeipa/freeipa/pull/2574
- Issue assigned to frenaud (was: fbarreto)

2 years ago

master:

  • 55c0a93 ipatests: fix test_replica_uninstall_deletes_ruvs

ipa-4-7:

  • 1f6bed4 ipatests: fix test_replica_uninstall_deletes_ruvs

ipa-4-6:

  • 1f19439 ipatests: fix test_replica_uninstall_deletes_ruvs

Metadata Update from @frenaud:
- Issue close_status updated to: fixed
- Issue status updated to: Closed (was: Open)

2 years ago

Login to comment on this ticket.

Metadata
Attachments 1
Attached 2 years ago View Comment