#2343 Proposal: Block (stable release) bodhi updates when they fail dist.rpmdeplint
Closed: Rejected 2 years ago by psabata. Opened 2 years ago by decathorpe.

Over the past few months, I've reported dozens of FTI issues caused by broken dependencies, either on bodhi updates if they were still in testing, or with bugzillas if it was too late for the former. Even worse, a lot of these affect stable releases. Most instances were caught by my fedora repo health check.

Can we discuss blocking bodhi updates if they fail the dist.rpmdeplint task, at least for stable branches, where this is 99% a real bug? Maybe even for rawhide, where stuff that introduces broken deps should be done with on-demand side-tags and submitted all at once, anyway ...

I think things like this were the original purpose of update gating, right? So it shouldn't be too hard to wire it up so that a FAIL result for dist.rpmdeplint blocks the update without a test result waiver.


Does this check reverse deps (other packages no longer install) or just direct (the updated package does not install)? The first is much more useful, but also probably controversial to gate on now. The latter is also useful while at the same time I think it should be a no brainer: not letting such packages get in is a good thing.

@churchyard According to the docs, the taskotron task only checks "direct" dependencies right now, though it claims that the underlying check supports "reverse" dependencies as well, but that has not been wired up yet:

https://fedoraproject.org/wiki/Taskotron/Tasks/dist.rpmdeplint#Known_deficiencies

There are multiple issues with current task-rpmdeplint:
1. There are known cases where it produces incorrect results, most notably with multilib packages. See https://pagure.io/taskotron/task-rpmdeplint/issue/6 . This is one of the reasons why we kept it just as an informational test all that time, its reliability hasn't been high enough.
2. rpmdeplint itself is currently unmaintained and having considerable technical debt (e.g. unaware about modules). Fedora QA team has pushed some changes to make it Python3-compatible, because there was no one else to take care of it, but we don't intend to maintain it and develop it.
3. It's not scheduled against Rawhide due to known issues, just stable Fedora releases.
4. If there is an infra error, we have no way of rescheduling the job (and definitely not as a package maintainer action).
5. It currently runs in Taskotron, and Taskotron will be EOL soon (in a few months). Everything is supposed to be running in Fedora CI now. @tflink and @jskladan are working on having generic tests support in Fedora CI, and they're targetting rpminspect as the first test. It that works out, we'll have a look at porting existing Taskotron tests (perhaps including rpmdeplint) over to Fedora CI, but no guarantees here.

I appreciate the proposal, but sadly this doesn't seem to be the right time nor the right project to make this change :confused:

thanks for your response, @kparal

Though I am a bit confused - taskotron is going away? That's news to me ... I find the test results that are displayed for bodhi updates extremely useful.

Though I am a bit confused - taskotron is going away? That's news to me ... I find the test results that are displayed for bodhi updates extremely useful.

Yes, it has been on life support the whole last year or so. I'm glad you find it useful. It's supposed to be gone with the Fedora Infra datacenter move. Once we'll figure out the exact date, we'll announce it ( https://pagure.io/taskotron/issue/279 ). There is a lot of effort being put into Fedora CI and there's little reason to keep several similar tools around. We hope the transition period will be short.

Killing taskotron before we have a reliable replacement seems like we are making the contributor experience and overall Fedora quality worse for no gain. But I realize that Fedora CI is an objective, and that somehow makes it more important. This is making me very sad.

Just so we have some data about this issue (only counting 2020, so far):

  • 5 unannounced / accidental SONAME bumps (that were caught and reported on the devel list) → about one issue per week (impacting a lot of packages)
  • filed 14 bugs for new(!) FTI / broken dependencies in stable releases and rawhide → about one issue every three days
  • commented on 8 bodhi updates in -testing that introduced new(!) broken dependencies → about one issue per week

So, at least, I'm not imagining things ...

BTW, this doesn't account for all the stuff that's already broken. I don't have the time, nor the energy to deal with pre-existing issues - filing bug reports and bodhi comments for new brokenness is enough work ...

Metadata Update from @psabata:
- Issue tagged with: meeting

2 years ago
  * AGREED: Close for now; as soon as something is ready to enable, ask
    us again about that specific thing (+6, 0, -0)  (contyk, 16:42:31)

Metadata Update from @psabata:
- Issue untagged with: meeting
- Issue close_status updated to: Rejected
- Issue status updated to: Closed (was: Open)

2 years ago

Can we revaluate this proposal?

There are multiple issues with current task-rpmdeplint:
1. There are known cases where it produces incorrect results, most notably with multilib packages. See https://pagure.io/taskotron/task-rpmdeplint/issue/6 . This is one of the reasons why we kept it just as an informational test all that time, its reliability hasn't been high enough.

Multilib checks were disabled. That's not perfect, but lowers a number of false negatives significantly. I believe that the only affected packages are those which intentionally have cross-architecture dependency like Wine.

  1. rpmdeplint itself is currently unmaintained and having considerable technical debt (e.g. unaware about modules). Fedora QA team has pushed some changes to make it Python3-compatible, because there was no one else to take care of it, but we don't intend to maintain it and develop it.

No comment.

  1. It's not scheduled against Rawhide due to known issues, just stable Fedora releases.

It's actually the opposite. It is running in Rawhide and I asked enabling in F36 too https://pagure.io/fedora-ci/general/issue/320.

  1. If there is an infra error, we have no way of rescheduling the job (and definitely not as a package maintainer action).

Packagers can reschedule the tests (bodhi updates trigger-tests) . Though all the tests again. Not rpmdeplint only.

  1. It currently runs in Taskotron, and Taskotron will be EOL soon (in a few months). Everything is supposed to be running in Fedora CI now. @tflink and @jskladan are working on having generic tests support in Fedora CI, and they're targetting rpminspect as the first test. It that works out, we'll have a look at porting existing Taskotron tests (perhaps including rpmdeplint) over to Fedora CI, but no guarantees here.

I guess this was resolved because two years later the test is still running.

We would need to resolve this issue first: https://pagure.io/fedora-ci/general/issue/155

I think a good first step would be to make it easy for package maintainers to opt-in to this (e.g. by documenting how to do this and advertising it). I'm in favor of this idea in principle, but I think we would need to run a trial with early adopters first to get an idea of how this would work in practice.

Documentation is at https://docs.fedoraproject.org/en-US/ci/gating/#_enable_gating. Though I would discourage enabling the test explicitly per component now because people tends to push Rawhide commits to older Fedoras and there the test is not supported right now https://pagure.io/fedora-ci/general/issue/320.

I think first we should enable support in all Fedoras. I agree that we should resolve performance issues and watch for false negatives and positives first. The problem is that nobody does that.

I can only say that I crosscheck complains for "announced soname change" I'm seeing recently in devel list with the rpmdeplint test output and it seems that the test works quite well.

What I don't know are false negatives. Yesterday, I noticed https://pagure.io/fedora-ci/general/issue/325.

Login to comment on this ticket.

Metadata