There have been ongoing problems with ELN contributions to Fedora that disable tests or enable vendoring under %if 0%{?rhel} conditionals. The purported goal of these PRs is to remove dependencies from the ELN buildroot. Other than the high rate at which these PRs are filed, the main problem is the way in which the developer of these PRs responds to feedback. They are often hostile when asked to apply changes or follow guidelines and best practices. They amp up the urgency of PRs with comments about how important it is to remove X packages from the ELN buildroot and claim that they'll address feedback later.

This is very much a mischaracterization of the response. At various times, we have requested that we merge an urgently-required fix prior to addressing unrelated issues. In the specific case that spawned this ticket, the request was essentially to implement a new series of bundling tools and macros. This is absolutely worth doing and should be tracked. I agree with Yaakov however that it shouldn't block the immediate fix.

I've seen PRs that are immediately merged using provenpackager privileges with no review.

Citation needed. I can't think of any such event. There have been times where we have merged changes after a couple of weeks without a response, but that's hardly "immediately".

This is not conducive to a proper code review process and is disrespectful of maintainer time.

Respect works both ways. If someone is providing solutions and a maintainer is ignoring them, that's hardly respectful either.

Sometimes, a packager does not wish to take on extensive changes to enable vendoring that'll add to their maintenance burden. In this case, the solution is apparently to create a separate eln branch. ELN developers have been asked to set a formal policy for these branches and to state who will be responsible for their maintenance.

This one is a completely legitimate concern that I have been trying very hard to resolve. The reality is that there's a huge gap between what should be happening here and what is happening.

From my perspective, when a branch like this has to be created for valid reasons, it should be the responsibility of the team at Red Hat that will be maintaining the package once it ships in a RHEL release. As a whole, RHEL management agrees with this.

Logistically, this isn't happening and that's a problem I'm working to resolve within Red Hat. I've not been sufficiently transparent around this effort, which is very much my fault. I will try to improve on this.

https://src.fedoraproject.org/rpms/python-rpds-py/pull-request/1 is one example, but this is a more pervasive problem that the python-sig has been dealing with over the past months. Here, the ELN developer was asked to include bundled(...) Provides when adding vendored dependencies to a package, and they claimed that Fedora ELN doesn't have to follow Fedora Packaging Guidelines and that they'd deal with it later. Bundling the dependencies that are not available is one thing, but not following basic guidelines about bundling/licensing/compliance/metadata is something else. These "we'll fix it later" PRs take up Fedora maintainers' time and lower the quality of the distribution.

See above: we absolutely agree that this is something that should happen. The sticking point is whether a multi-week engineering effort needs to block what would otherwise be a nearly trivial fix for an urgent issue. This seems like a clear case of perfect being the enemy of "good enough".

I'd like to see the following issues addressed:

1. There should be a formal policy about eln branches. Who is responsible for their maintenance and in what cases should they be created?

As I said above, I am working on this. I described in broad strokes what it should be and I'm trying to get that formalized properly.

1. The ways in which ELN diverges from Packaging Guidelines need to be documented so Fedora maintainers can properly review changes.

I think this was an overstatement on Yaakov's part. Our packaging guidelines for RHEL are meant to remain as close to Fedora's as is reasonably possible. And I agree with you, @gotmax23 : bundling needs to be handled with appropriate bundled(foo) annotations.

The disagreement is on whether it's acceptable for some non-critical and/or non-technical policies to have their implementations delayed in favor of expediency. (I'm not asking for a blanket answer here; different instances will have different needs.)

1. These changes are limited to packages that will actually end up as part of RHEL to avoid piling up cruft that will add to maintenance burden and affect EPEL packages that we can afford to package in the proper way.

I'm not really sure how to parse this statement, since ELN is literally the set of packages that will end up in RHEL. The issue in question was specifically triggered by a package change in Fedora resulting in too many packages being pulled into RHEL, so we provided a fix that was... less than well-received.

1. ELN developers are willing to respond to feedback and make changes.

That has never been a problem, though we are only human. Feedback that doesn't sound like it's being offered in good faith can leave us a bit grouchy. We're all on the same side here: we all want Fedora and its descendant ecosystem to be successful.

/cc @sgallagh @yselkowitz

There have been ongoing problems with ELN contributions to Fedora that disable tests or enable vendoring under %if 0%{?rhel} conditionals. The purported goal of these PRs is to remove dependencies from the ELN buildroot. Other than the high rate at which these PRs are filed, the main problem is the way in which the developer of these PRs responds to feedback. They are often hostile when asked to apply changes or follow guidelines and best practices. They amp up the urgency of PRs with comments about how important it is to remove X packages from the ELN buildroot and claim that they'll address feedback later.

This is very much a mischaracterization of the response. At various times, we have requested that we merge an urgently-required fix prior to addressing unrelated issues. In the specific case that spawned this ticket, the request was essentially to implement a new series of bundling tools and macros. This is absolutely worth doing and should be tracked. I agree with Yaakov however that it shouldn't block the immediate fix.

The immediate fix was to add the bundled provides manually. They were requested by the package maintainer yet the PR was merged into the eln branch without them.

I've seen PRs that are immediately merged using provenpackager privileges with no review.

Citation needed. I can't think of any such event. There have been times where we have merged changes after a couple of weeks without a response, but that's hardly "immediately".

https://src.fedoraproject.org/rpms/python-requests/pull-request/22
https://src.fedoraproject.org/rpms/python-lxml/pull-request/25#comment-144485

This is not conducive to a proper code review process and is disrespectful of maintainer time.

Respect works both ways. If someone is providing solutions and a maintainer is ignoring them, that's hardly respectful either.

While this statement is technically correct (which is the best kind of correct), I have no idea what it relates to. We are not ignoring solutions here, we are trying to review changes before they land and offer constructive feedback.

...

https://src.fedoraproject.org/rpms/python-rpds-py/pull-request/1 is one example, but this is a more pervasive problem that the python-sig has been dealing with over the past months. Here, the ELN developer was asked to include bundled(...) Provides when adding vendored dependencies to a package, and they claimed that Fedora ELN doesn't have to follow Fedora Packaging Guidelines and that they'd deal with it later. Bundling the dependencies that are not available is one thing, but not following basic guidelines about bundling/licensing/compliance/metadata is something else. These "we'll fix it later" PRs take up Fedora maintainers' time and lower the quality of the distribution.

See above: we absolutely agree that this is something that should happen. The sticking point is whether a multi-week engineering effort needs to block what would otherwise be a nearly trivial fix for an urgent issue. This seems like a clear case of perfect being the enemy of "good enough".

I respectfully disagree. Not following Fedora packaging guidelines and merging the changes despite negative feedback from 3 different packagers is not "good enough".

1. ELN developers are willing to respond to feedback and make changes.

That has never been a problem, though we are only human. Feedback that doesn't sound like it's being offered in good faith can leave us a bit grouchy. We're all on the same side here: we all want Fedora and its descendant ecosystem to be successful.

I see this as a recurring, long-term problem with the ELN-motivated pull requests. I always provide feedback in good faith yet it is repeatedly played down or outright disregarded. Quietly assuming my feedback is not provided in good faith is not an excuse to ignore it.

There have been ongoing problems with ELN contributions to Fedora that disable tests or enable vendoring under %if 0%{?rhel} conditionals. The purported goal of these PRs is to remove dependencies from the ELN buildroot. Other than the high rate at which these PRs are filed, the main problem is the way in which the developer of these PRs responds to feedback. They are often hostile when asked to apply changes or follow guidelines and best practices. They amp up the urgency of PRs with comments about how important it is to remove X packages from the ELN buildroot and claim that they'll address feedback later.

This is very much a mischaracterization of the response. At various times, we have requested that we merge an urgently-required fix prior to addressing unrelated issues. In the specific case that spawned this ticket, the request was essentially to implement a new series of bundling tools and macros. This is absolutely worth doing and should be tracked. I agree with Yaakov however that it shouldn't block the immediate fix.

I feel there's often pressure to merge these ELN PRs as quickly as possible much more so than with other PRs. We were not asking to to develop a whole new set of bundling tools. Something like https://src.fedoraproject.org/rpms/netavark/blob/40294de062d4e37f6a6b434d49a6cf7f0ba50747/f/netavark.spec#_47 would've taken 30 seconds to generate. I suggested as much.

I've seen PRs that are immediately merged using provenpackager privileges with no review.

Citation needed. I can't think of any such event. There have been times where we have merged changes after a couple of weeks without a response, but that's hardly "immediately".

https://src.fedoraproject.org/rpms/python-lxml/pull-request/25 is one.

This is not conducive to a proper code review process and is disrespectful of maintainer time.

Respect works both ways. If someone is providing solutions and a maintainer is ignoring them, that's hardly respectful either.

Multiple people provided feedback that was ignored, and the PR was merged anyways despite outstanding issues that were even unrelated to bundling.

Sometimes, a packager does not wish to take on extensive changes to enable vendoring that'll add to their maintenance burden. In this case, the solution is apparently to create a separate eln branch. ELN developers have been asked to set a formal policy for these branches and to state who will be responsible for their maintenance.

This one is a completely legitimate concern that I have been trying very hard to resolve. The reality is that there's a huge gap between what should be happening here and what is happening.

From my perspective, when a branch like this has to be created for valid reasons, it should be the responsibility of the team at Red Hat that will be maintaining the package once it ships in a RHEL release. As a whole, RHEL management agrees with this.

Logistically, this isn't happening and that's a problem I'm working to resolve within Red Hat. I've not been sufficiently transparent around this effort, which is very much my fault. I will try to improve on this.

Thanks for working on the problem. I find it worrying that these branches continue to proliferate with nobody being responsible for them.

https://src.fedoraproject.org/rpms/python-rpds-py/pull-request/1 is one example, but this is a more pervasive problem that the python-sig has been dealing with over the past months. Here, the ELN developer was asked to include bundled(...) Provides when adding vendored dependencies to a package, and they claimed that Fedora ELN doesn't have to follow Fedora Packaging Guidelines and that they'd deal with it later. Bundling the dependencies that are not available is one thing, but not following basic guidelines about bundling/licensing/compliance/metadata is something else. These "we'll fix it later" PRs take up Fedora maintainers' time and lower the quality of the distribution.

See above: we absolutely agree that this is something that should happen. The sticking point is whether a multi-week engineering effort needs to block what would otherwise be a nearly trivial fix for an urgent issue. This seems like a clear case of perfect being the enemy of "good enough".

Again, nobody asked for a multi week engineering effort, although I'd dispute that the automation proposed would take that long to implement. Simply pasting the Provides into the specfile like we've been doing for years would've solved the immediate problem.

I'd like to see the following issues addressed:

1. There should be a formal policy about eln branches. Who is responsible for their maintenance and in what cases should they be created?

As I said above, I am working on this. I described in broad strokes what it should be and I'm trying to get that formalized properly.

That's good to hear. I'm glad that it's already being worked on.

1. The ways in which ELN diverges from Packaging Guidelines need to be documented so Fedora maintainers can properly review changes.

I think this was an overstatement on Yaakov's part. Our packaging guidelines for RHEL are meant to remain as close to Fedora's as is reasonably possible. And I agree with you, @gotmax23 : bundling needs to be handled with appropriate bundled(foo) annotations.

Right.

The disagreement is on whether it's acceptable for some non-critical and/or non-technical policies to have their implementations delayed in favor of expediency. (I'm not asking for a blanket answer here; different instances will have different needs.)

I would've been happy to help come up with a workable solution (see the above example), but it seemed the main goal was to get this PR merged as quickly as possible.

1. These changes are limited to packages that will actually end up as part of RHEL to avoid piling up cruft that will add to maintenance burden and affect EPEL packages that we can afford to package in the proper way.

I'm not really sure how to parse this statement, since ELN is literally the set of packages that will end up in RHEL. The issue in question was specifically triggered by a package change in Fedora resulting in too many packages being pulled into RHEL

One example is https://src.fedoraproject.org/rpms/python-libcst/c/7be0eb74034cc364f9908ed194c128937f221c6c?branch=rawhide which is used for processing python syntax trees by things like code formatters and is very unlikely to end up in RHEL. Maybe this is overstated, but it seems some issues are being handled at the wrong part of the dependency tree.

we provided a fix that was... less than well-received.

It was poorly received, because it was not compliant with the guidelines and our valid review feedback was dismissed.

1. ELN developers are willing to respond to feedback and make changes.

That has never been a problem, though we are only human. Feedback that doesn't sound like it's being offered in good faith can leave us a bit grouchy. We're all on the same side here: we all want Fedora and its descendant ecosystem to be successful.

I don't think we're acting in bad faith. I might've been a little grumpier than usual, partially due to recovering from surgery and partially due to past issues with this maintainer, but everyone involved in reviewing that PR genuinely cares about Fedora and wants to make sure we are merging high quality changes. It doesn't feel great when feedback is dismissed or people claim they're above the rules. Hopefully, we can all work together more productively and respectfully in the future.

Since the cause of this ticket seems to have involved Rust dependencies, I'll add a few comments here (since I basically ended up being "Mr. Rust" a while ago, or "dictator", as some have called me when they didn't think I was around).

There are already a handful (I think ~10) Rust-ish packages that are getting pulled into ELN, and I've been trying to smooth over the issues that are caused by the two often diametrically opposed approaches to packaging these kinds of projects in Fedora and RHEL.

Most of these packages now have separate ELN branches, because keeping the conditionals in the Fedora packages is making package maintenance much harder. This is because Rust packages basically need to be regenerated with rust2rpm for every new version (which I hope will eventually be a much smaller issue once we can switch to dynamic spec snippet generation with the latest RPM versions).

Without a separate eln branch, this means that all manual changes - including the vendoring machinery for ELN / RHEL - need to be manually re-applied every time, and the vendor tarball needs to be generated manually. This is very tedious, and potentially error-prone, which is why I opted for separate eln branches for all Rust packages where I am (effectively) the primary maintainer. I've been keeping the eln branches of these packages updated myself, since there's no RHEL developers in the loop (yet?) - this now includes core packages of the distro, like rpm-sequoia, which are installed on every system and part of the default buildroot.

What's making things worse is that RHEL does not support some RPM macros for Rust packaging that are now supported on all current branches of Fedora and even on EPEL 9 - either they are missing completely, or they offer less functionality in RHEL. For example, the %cargo_license and %cargo_license_summarymacros that help with generating a license breakdown for all statically linked components are missing in RHEL.

In this case, I don't know why the macros haven't been backported into rust-toolset (or whatever the RHEL package is called that provides these macros), since they have no external dependencies other than cargo (obviously already pulled in for builds that use cargo), /usr/bin/sed, and /usr/bin/sort (both coreutils?).

As for providing macros / generators for generating bundled(...) provides for vendored Rust crates automatically, the Rust SIG has never been approached about implementing something like that, but given that this could be built similarly to the existing %cargo_license{,_summary} macros, I think it would be possible to have a working prototype (that works similarly to the Go generator) within a few hours.

I don't know how long it would take to get new RPM macros / generators into RHEL proper, but getting them into Fedora (and, by extension, ELN) would be straightforward (since I am now both upstream and downstream packager for all projects that would be involved).

To be clear, I'm not strictly complaining about using vendored dependencies (this is, after all, also allowed in Fedora in certain circumstances), but about missing Provides: bundled(...) for them, and for missing license breakdowns. I realize that getting technical issues resolved quickly is often more important than doing things the right way :tm: - but this is not a new issue. The first PRs for dropping Rust dependencies on RHEL were filed over 4 months ago.

I can spend some time working on tooling support to make Rust packages more in-line with packaging guidelines in ELN (i.e. using vendored dependencies, but correctly declaring Provides for them, and listing licences correctly), but last time I offered to work on making Rust packaging more palatable for ELN / RHEL, things went nowhere. That might just have been because the developers who are responsible for ELN aren't even sure what tooling support they'd need - because they're not Rust packagers, after all - but it also didn't seem to have been a priority, either.

Ok, I need to stop rambling now, but I hope this is useful context for this discussion, and maybe helps as a starting point for moving things forward.

PS: One thing that does confuse me is that we keep getting told that "ELN is Fedora", but sometimes it really doesn't feel that way - do Fedora Packaging Guidelines apply to ELN now, or don't they? If the rules do diverge, this definitely needs to be documented somewhere.

The immediate fix was to add the bundled provides manually. They were requested by the package maintainer yet the PR was merged into the eln branch without them.

OK, that's a mistake/miscommunication. I agree that the bundled(foo) stuff should have been manually added (and I thought it was; I thought the sticking point was with automation around that). I'll ask @yselkowitz to send a follow-up patch to correct that.

I've seen PRs that are immediately merged using provenpackager privileges with no review.

Citation needed. I can't think of any such event. There have been times where we have merged changes after a couple of weeks without a response, but that's hardly "immediately".

https://src.fedoraproject.org/rpms/python-requests/pull-request/22

OK, I either missed or forgot about that one; it was months ago. That said, it was about as trivial a patch as I can imagine (just flipping an existing bcond default). However...

https://src.fedoraproject.org/rpms/python-lxml/pull-request/25#comment-144485

Yeah, this one was unquestionably wrong to merge without review.

Those are both from months ago, though. I'm going to assume that this was a lesson learned, rather than a pattern of behavior.

This is not conducive to a proper code review process and is disrespectful of maintainer time.

Respect works both ways. If someone is providing solutions and a maintainer is ignoring them, that's hardly respectful either.

While this statement is technically correct (which is the best kind of correct), I have no idea what it relates to. We are not ignoring solutions here, we are trying to review changes before they land and offer constructive feedback.

That was just a follow up to the previous note that many of our MRs go entirely ignored, which is also frustrating. It's not directly relevant here other than to note that frustrations can sometimes pile up outside of the viewer's frame of reference.

...

https://src.fedoraproject.org/rpms/python-rpds-py/pull-request/1 is one example, but this is a more pervasive problem that the python-sig has been dealing with over the past months. Here, the ELN developer was asked to include bundled(...) Provides when adding vendored dependencies to a package, and they claimed that Fedora ELN doesn't have to follow Fedora Packaging Guidelines and that they'd deal with it later. Bundling the dependencies that are not available is one thing, but not following basic guidelines about bundling/licensing/compliance/metadata is something else. These "we'll fix it later" PRs take up Fedora maintainers' time and lower the quality of the distribution.

See above: we absolutely agree that this is something that should happen. The sticking point is whether a multi-week engineering effort needs to block what would otherwise be a nearly trivial fix for an urgent issue. This seems like a clear case of perfect being the enemy of "good enough".

I respectfully disagree. Not following Fedora packaging guidelines and merging the changes despite negative feedback from 3 different packagers is not "good enough".

Fair enough. As I said above, I misunderstood one of the sticking points (the bundling annotations). You're correct about that point and it really should have been addressed. Let's fix that.

1. ELN developers are willing to respond to feedback and make changes.

That has never been a problem, though we are only human. Feedback that doesn't sound like it's being offered in good faith can leave us a bit grouchy. We're all on the same side here: we all want Fedora and its descendant ecosystem to be successful.

I see this as a recurring, long-term problem with the ELN-motivated pull requests. I always provide feedback in good faith yet it is repeatedly played down or outright disregarded. Quietly assuming my feedback is not provided in good faith is not an excuse to ignore it.

That... came out wrong in exactly the way I was trying to capture.

It's easy (especially in the heat of the moment) to phrase things in a way that come across wrong. My intent was to explain that the listener/reader can sometimes misinterpret what is in good faith and what is not. The longer the back-and-forth, the more frustrating (and less forgiving) we can all get.

I think in some of these instances, what is intended as a sincere effort to improve quality can also come across as nitpicking for its own sake, interpreted by a frustrated mind as an intentional delaying tactic.

It can be really hard to get out of one's own way, sometimes.

...
I think I covered the earlier parts of this in my reply to Miro. Please correct me if I missed something.

Sometimes, a packager does not wish to take on extensive changes to enable vendoring that'll add to their maintenance burden. In this case, the solution is apparently to create a separate eln branch. ELN developers have been asked to set a formal policy for these branches and to state who will be responsible for their maintenance.

This one is a completely legitimate concern that I have been trying very hard to resolve. The reality is that there's a huge gap between what should be happening here and what is happening.

From my perspective, when a branch like this has to be created for valid reasons, it should be the responsibility of the team at Red Hat that will be maintaining the package once it ships in a RHEL release. As a whole, RHEL management agrees with this.

Logistically, this isn't happening and that's a problem I'm working to resolve within Red Hat. I've not been sufficiently transparent around this effort, which is very much my fault. I will try to improve on this.

Thanks for working on the problem. I find it worrying that these branches continue to proliferate with nobody being responsible for them.

Or worse, that I or Yaakov end up being responsible for all of them. I like to occasionally sleep!

https://src.fedoraproject.org/rpms/python-rpds-py/pull-request/1 is one example, but this is a more pervasive problem that the python-sig has been dealing with over the past months. Here, the ELN developer was asked to include bundled(...) Provides when adding vendored dependencies to a package, and they claimed that Fedora ELN doesn't have to follow Fedora Packaging Guidelines and that they'd deal with it later. Bundling the dependencies that are not available is one thing, but not following basic guidelines about bundling/licensing/compliance/metadata is something else. These "we'll fix it later" PRs take up Fedora maintainers' time and lower the quality of the distribution.

See above: we absolutely agree that this is something that should happen. The sticking point is whether a multi-week engineering effort needs to block what would otherwise be a nearly trivial fix for an urgent issue. This seems like a clear case of perfect being the enemy of "good enough".

Again, nobody asked for a multi week engineering effort, although I'd dispute that the automation proposed would take that long to implement. Simply pasting the Provides into the specfile like we've been doing for years would've solved the immediate problem.

As I noted in my other reply, I thought we were being asked to implement an automated Provides: system like Node.js has. Which is something that we absolutely can and probably should do, but it seemed like overkill for the specific MR. I now realize I missed the part where we were missing the manual Provides.

I'd like to see the following issues addressed:

1. There should be a formal policy about eln branches. Who is responsible for their maintenance and in what cases should they be created?

As I said above, I am working on this. I described in broad strokes what it should be and I'm trying to get that formalized properly.

That's good to hear. I'm glad that it's already being worked on.

1. The ways in which ELN diverges from Packaging Guidelines need to be documented so Fedora maintainers can properly review changes.

I think this was an overstatement on Yaakov's part. Our packaging guidelines for RHEL are meant to remain as close to Fedora's as is reasonably possible. And I agree with you, @gotmax23 : bundling needs to be handled with appropriate bundled(foo) annotations.

Right.

The disagreement is on whether it's acceptable for some non-critical and/or non-technical policies to have their implementations delayed in favor of expediency. (I'm not asking for a blanket answer here; different instances will have different needs.)

I would've been happy to help come up with a workable solution (see the above example), but it seemed the main goal was to get this PR merged as quickly as possible.

I think we failed to communicate the reason why, here. We're currently in the middle of a day-by-day slip as we try to get the big CentOS Stream 10 import started. (It was originally scheduled for the first week of August, but a number of blocking issues crept up.)

We're very close now to being ready to do the initial import, but all of a sudden this one package caused Content Resolver to leap by 1200 packages. Due to limitations in the resolver, we don't have great historical data to make sure we know what to exclude, so we rushed to trim those back out.

(There's a lot of technical, political and economic reasons, but it's quite painful to try to remove packages after they are imported.)

1. These changes are limited to packages that will actually end up as part of RHEL to avoid piling up cruft that will add to maintenance burden and affect EPEL packages that we can afford to package in the proper way.

I'm not really sure how to parse this statement, since ELN is literally the set of packages that will end up in RHEL. The issue in question was specifically triggered by a package change in Fedora resulting in too many packages being pulled into RHEL

One example is https://src.fedoraproject.org/rpms/python-libcst/c/7be0eb74034cc364f9908ed194c128937f221c6c?branch=rawhide which is used for processing python syntax trees by things like code formatters and is very unlikely to end up in RHEL. Maybe this is overstated, but it seems some issues are being handled at the wrong part of the dependency tree.

We are not subject matter experts on every package in the distribution, obviously :)

We don't always know what each package does, but we have tooling that helps us figure out where we can cut dependencies that result in large swaths of the pile falling off. In this particular case, my guess is that this package was one of the last holdouts for a major node in the dep tree.

I don't have a specific answer for this case (it was seven months ago, too).

we provided a fix that was... less than well-received.

It was poorly received, because it was not compliant with the guidelines and our valid review feedback was dismissed.

Fair enough.

1. ELN developers are willing to respond to feedback and make changes.

That has never been a problem, though we are only human. Feedback that doesn't sound like it's being offered in good faith can leave us a bit grouchy. We're all on the same side here: we all want Fedora and its descendant ecosystem to be successful.

I don't think we're acting in bad faith. I might've been a little grumpier than usual, partially due to recovering from surgery and partially due to past issues with this maintainer, but everyone involved in reviewing that PR genuinely cares about Fedora and wants to make sure we are merging high quality changes. It doesn't feel great when feedback is dismissed or people claim they're above the rules. Hopefully, we can all work together more productively and respectfully in the future.

See my last note on my reply to Miro. I fell into the exact hole I was trying to describe.

"dictator", as some have called me when they didn't think I was around).

At least, they called you a benevolent dictator :).

That was just a follow up to the previous note that many of our MRs go entirely ignored, which is also frustrating. It's not directly relevant here other than to note that frustrations can sometimes pile up outside of the viewer's frame of reference.

From personal experience, I know this is a very frustrating problem, but in these cases, the PRs were filed against well maintained packages with active maintainers.

Yeah, this one was unquestionably wrong to merge without review.

Those are both from months ago, though. I'm going to assume that this was a lesson learned, rather than a pattern of behavior.

Well, it seems that pattern is still ongoing. https://src.fedoraproject.org/rpms/python-rpds-py/pull-request/1 where this happened was the impetus of this ticket. Please avoid that in the future.

I think we failed to communicate the reason why, here. We're currently in the middle of a day-by-day slip as we try to get the big CentOS Stream 10 import started. (It was originally scheduled for the first week of August, but a number of blocking issues crept up.)

These urgent PRs have been a problem for multiple months, not just here, but I guess that's besides the point.

Other than that, I think the three remaining problems are:

1. Documenting eln branch policy
2. Documenting Packaging Guidelines differences (even if the only differences are that disabling tests and vendoring is allowed in more instances).
3. Trying to make these PRs less disruptive. "TASK: remove tox from buildroot #126" which was brought up earlier in the thread were quite disruptive as 20 packages had to be fixed within less than two weeks.

I guess 1. should be documented somewhere in https://docs.fedoraproject.org/en-US/eln/ and 2. should be its in own page in the Packaging Guidelines. 3. is probably a planning issue within the ELN team. For the rust bundled(...) Provides generator, someone should file a ticket in https://pagure.io/fedora-rust/rust-packaging instead of discussing here. I implemented something similar for Go, so let me know if I can help in any way.

What do you think?

Those are both from months ago, though. I'm going to assume that this was a lesson learned, rather than a pattern of behavior.

That's unfortunately very hard to say, considering there was no follow-up response to my comments. You asked for a citation, I provided it.

However, considering what happened today in https://src.fedoraproject.org/rpms/python-rpds-py/pull-request/1 I would be reluctant to consider this a problem of the past.

We are not subject matter experts on every package in the distribution, obviously :)

Actually, I'd say that's one more reason to actually consider feedback from the subject matter experts.

I think in some of these instances, what is intended as a sincere effort to improve quality can also come across as nitpicking for its own sake, interpreted by a frustrated mind as an intentional delaying tactic.

I don't know how to respond to this. I realize you probably did not mean to, but this can be interpreted by my frustrated mind as trying to trivialize this problem :(

This was discussed during today's FESCo meeting:
ACTION: sgallagh to preparate an update for the ELN documentation

I haven't updated the ELN documentation yet because I've actually started getting traction on the internal policy change regarding eln branch ownership. I'll try to get it all updated at once in the next week or two.

@sgallagh Where are we at here?

I've pitched this internally and got general buy-in, but (as usual), things are stuck on minutiae at the minute.

@sgallagh Any progress?

Can we at least document the Packaging Guidelines part if the eln branch part is still in the works?

I thought we had documented the branch policy, and it turns out we had, but it's not linked anywhere properly.

The direct link is https://docs.fedoraproject.org/en-US/eln/eln-branch/

I'll try to figure out why the index for it is broken.

Found and fixed that; it will appear in the side navigation bar within 30 minutes.

Regarding "2. Documenting Packaging Guidelines differences (even if the only differences are that disabling tests and vendoring is allowed in more instances).":

I don't know what, specifically, we need to document here. Nothing happening here is impermissible in the standard guidelines. The only difference is that packages slated for ELN have a higher incidence of these things for the purposes of keeping the package count down in ELN/RHEL. I can add some justifications in the ELN documentation anyway.

The main reason for streamlining the packages is for license and export compliance reasons: every package in RHEL (regardless of whether it gets shipped to end-users) has to be evaluated individually. We have tooling internally to help with this, but as @churchyard can no doubt attest, it still takes a considerable amount of human time to review and validate those licenses. Fedora has a more lax policy here for a variety of reasons that you'd need to ask an actual lawyer about. But for everything that goes into RHEL, there's a mountain of paperwork to do. For obvious reasons, that explanation doesn't really provide a strong argument for non-RHT maintainers in Fedora.

That all said, I think a large part of the problem has been on my end. I've done a poor job of communicating our internal milestones out to the public (in some cases, this has been due to legal restrictions around "forward-looking statements", but not so many as to be a convenient excuse). The result of that has been a number of merge requests that have landed late and been merged too quickly. I know the correct solution here: be more open about our deadlines and schedules. I'm going to try my best to do that. Among other things, I definitely owe another widely-distributed status update with upcoming plans; I'll try to put that out within the next week.

To summarize:
- eln branch policy is at https://docs.fedoraproject.org/en-US/eln/eln-branch/
- We clarified that the Fedora Packaging Guidelines also apply to eln branches, and eln packaging just make some different choices to minimize dependencies.
- There is a commitment to improve communication around internal RHEL/ELN milestones and other choices.

I think this answers the four points that were raised in the initial ticket. Is anybody opposed to closing this?

I have only one open question. From the ELN SIG Meeting notes ( https://meetbot.fedoraproject.org/fedora-meeting/2023-09-22/eln.2023-09-22-16.00.log.html ):

#agreed Proposal 1: Fedora ELN will follow all published Fedora Packaging Guidelines, except where explicitly documented otherwise. (+4, 0, 0)

I couldn't find that documentation anywhere in the ELN docs ( https://docs.fedoraproject.org/en-US/eln/ ). I also asked in the ELN Matrix channel if those docs exist and where I can find them just now.

I have only one open question. From the ELN SIG Meeting notes ( https://meetbot.fedoraproject.org/fedora-meeting/2023-09-22/eln.2023-09-22-16.00.log.html ):

#agreed Proposal 1: Fedora ELN will follow all published Fedora Packaging Guidelines, except where explicitly documented otherwise. (+4, 0, 0)

I couldn't find that documentation anywhere in the ELN docs ( https://docs.fedoraproject.org/en-US/eln/ ). I also asked in the ELN Matrix channel if those docs exist and where I can find them just now.

Currently, we don't have any documented exceptions to the guidelines; we are trying to follow them as written. We probably should document the lack of exceptions explicitly though. Sorry about that.

That would be great, thank you! I think we can close this ticket then.

https://github.com/fedora-eln/eln-docs/pull/37

ELN SIG accepted the following:
Proposal 1: Fedora ELN will follow all published Fedora Packaging Guidelines, except where explicitly documented otherwise. (+4, 0, 0)

The pull request to clarify this in the ELN docs was just merged.

Let's close this ticket. I'll add a short announcement to todays' agenda.

And yet this keeps happening:

https://src.fedoraproject.org/rpms/python-urllib3/pull-request/38#comment-219866
https://src.fedoraproject.org/rpms/python-requests/pull-request/33#comment-219867

I don't know if a single event most of a year since the last one qualifies as "this keeps happening".

From the timing of things, I expect what happened is that he looked at the queue of merge requests that had been open a long time and just missed that these two had unaddressed comments.

I've reached out to Yaakov directly for comment.

Again, you are downplaying this issue. It matters to me and it makes me feel sad. I still haven't heard a response to my comments in the individual PRs linked previously, acknowledging that the merges violated the provenpackager policy. I keep hearing that I am imagining things.

It's ELN-motivated PRs merged by the submitter without approval, abusing provenpackager status, again.

It took ~1 month to get the %eln usage documented and approved via ELN SIG, but the PR suddenly needed to be merged immediately and could not wait for an input from any of the maintainers.

Perhaps I should have said "this is happening again" rather than "this keeps happening". Either way, after all that has been said here, this should not been happening.

Perhaps I should have said "this is happening again" rather than "this keeps happening". Either way, after all that has been said here, this should not be happening.

Again, you are downplaying this issue. It matters to me and it makes me feel sad. I still haven't heard a response to my comments in the individual PRs linked previously, acknowledging that the merges violated the provenpackager policy. I keep hearing that I am imagining things.

I'm not attempting to downplay it and I agree that the policy was violated in this instance. I never said you were imagining things. I said that I suspected that it was an honest mistake and I reached out to Yaakov to have him respond.

It's ELN-motivated PRs merged by the submitter without approval, abusing provenpackager status, again.

As I said, I think that the lack of approval in these two instances was likely a mistake or oversight. I prefer to give people the benefit of the doubt.

It took ~1 month to get the %eln usage documented and approved via ELN SIG, but the PR suddenly needed to be merged immediately and could not wait for an input from any of the maintainers.

A week went by between when the usage was documented and when it was merged. Again, I don't think that's "immediate", but I do agree that the lack of action on the comment request was incorrect.