As a developer, I want to have all misconfigured test cases to fail with a clear and visible test failure. The pytest_multihost plugin turns lack of configured resources into silently skipped test instead of a failed test. For example the test case test_caless.py::TestClientInstall::()::test_client_install::setup is currently skipped with message:
pytest_multihost
test_caless.py::TestClientInstall::()::test_client_install::setup
"Skipped: Not enough resources configured: Domain 0 not configured: {'type': 'IPA', 'hosts': {'master': 1, 'replica': 0, 'client': 1}}"
but PR-CI lists the test as green. There are multiple more test runs that are silently skipped, because the job doesn't have enough resources configured.
I propose to turn any misconfiguration into an error, so we are able to spot these issues easily. Otherwise we'd have to verify each and every test job manually.
master:
Metadata Update from @cheimes: - Issue close_status updated to: fixed - Issue status updated to: Closed (was: Open)
ipa-4-7:
Login to comment on this ticket.