Does it include checks which run through taskotron, for example dependency checks? rpmlint?

The results considered for the moment are defined in: https://infrastructure.fedoraproject.org/cgit/ansible.git/tree/roles/openshift-apps/greenwave/templates/configmap.yml

and basically boils down to:

• for push to updates-testing
  • dist.rpmdeplint must pass on the build
  • (For F26) tests ran by the Atomic CI pipeline must pass (for the packages being in specified list and having tests in their dist-git)
• for push to updates
  • dist.rpmdeplint must pass on the build
  • dist.abicheck must pass on the build except for a specific list of packages that are blacklisted (including libreoffice which make the check run out of memory for example) -- blacklist was provided by QA

The other checks aren't considered for gating.

@pingou so this doesn't affect rawhide, right?

@ignatenkobrain how rawhide would be affected by gating in bodhi?

I'd love to have rawhide be gated on test results but I don't think we're there atm :)

so this doesn't affect rawhide, right?

No, but @mohanboddu and I have been working on also using the greenwave policies to gate Rawhide. There is a rawhide policy in the greenwave config, but the releng tools haven't been adjusted to reference it yet.

• AGREED: Issue #1810: Let's flip the switch on January 15th: gating
  in Fedora is approved (+8,0,-0) (jforbes, 16:25:08)

Uh, dist.abicheck produces a lot of false positives on:

• libraries that are internal and that nothing should depend on (e.g., in QupZilla, package qupzilla),
• APIs explicitly documented as "private, can change at any version", as common in all Qt modules (e.g., in QtWebEngine, package qt5-qtwebengine).

My packages often fail dist.abicheck. It is absolutely not realistic to expect it to pass for all updates.

To @kkofler 's point: I have filed an issue on the abicheck task for this now - https://pagure.io/task-abicheck/issue/8

And while we're at it, a quick note on the current status of this fiasco^H^H^H^H^H^Hchallenging situation:

• The abicheck test is no longer gating for now, until that issue is resolved. I am going to do a bit more research into exactly how much of a problem it is - I'll look at the last 50 or 100 abicheck failures and see how many seem 'wrong'.
• The Greenwave policy has been updated to have no requirements for EPEL updates, so EPEL updates are no longer blocked.
• Some updates are blocked because a gating test did not run for a package in that update. We have a list of these. @pingou is commenting on the updates, and I will mail devel@ with the list. Affected maintainers can repush the update: this will re-trigger the task.rpmdeplint test, but reset the karma and the 7-day wait period clock. Some other tests cannot be retriggered this way, but task.rpmdeplint should be the only one that matters now. Or they can wait for the Greenwave changes described in the next two bullets, which provide a couple of other options.
• There is a PR for waiverdb to allow waiving the absence of a result (this is currently not possible). If that is merged, maintainers affected by a missing result would be able to waive this case. @ralph is planning to look at that PR today.
• As a possible more systemic medium-term mitigation for the problem than case-by-case repushes or waivers, @ralph is working on adjusting Greenwave to have a rule "test must pass or be absent", rather than the current rule "test must pass". This would allow us to adjust the policy to use that rule, so updates will be pushable so long as all blocking tests either pass or are absent, which means non-run tests are no longer a big problem.
• We intend to try to identify the various causes of non-run tests and ensure issues are filed for them. If we did implement the mitigation described above, then cnce those issues are resolved to such a degree that the taskotron folks are confident the blocking tests will just about always be run (whatever degree of confidence we decide on), the policy could be switched back to the tighter rule. That's for @tflink , @kparal , @jskladan etc.

The abicheck test is no longer gating for now, until that issue is resolved.

You assume there that this issue is fixable at all. I don't think it is, see:
https://pagure.io/task-abicheck/issue/8#comment-490378