I maintain dnsmasq package on Fedora. I had STI tests enabled, then my colleague create PR for tmt tests. But tests/tests.yaml stayed present. But we updated gating.yaml to accept only test ids, which only tmt produces.
In this state it passed tests in PR, but production build got stuck on gating. I think Zuul already runs almost all tests bodhi runs. It should gather all finished test ids and run check against gating.yaml if present. Should test whether it can pass into testing and can pass into stable, according to gating rules.
Example is current https://bodhi.fedoraproject.org/updates/FEDORA-2022-ac9daf8e4c:
Test were successful: fedora-ci.koji-build.tier0.functional Test result missing: fedora-ci.koji-build./plans/tier1-public.functional
I think test mapping to those ids should have also own check.
But gating.yaml contained: - !PassingTestCaseRule {test_case_name: fedora-ci.koji-build.tier0.functional} # STI tests
@mvadkert could gating tests be run by testing farm ?
@fbo we do not have a gating test for what he describes. He is basically asking for sanity testing gating.yaml and making sure it is correct with the test setup. Such test does not yet exist (even in downstream)
@pemensik thanks for the idea, the solution you propose is not very likely to happen. I believe we can create a tmt check for gating.yaml validity though. We have some prototype here:
https://gitlab.com/testing-farm/tests/-/blob/main/gating/sanity.fmf
But it was never finished :( Here is a downstream issue tracking in in Jira: https://issues.redhat.com/browse/TFT-1047
Login to comment on this ticket.