#1218 Before this starts causing us in QA serious headache there should be manatory description on copr repos
Closed None Opened 5 years ago by johannbg.

= phenomenon =

Before this starts causing us in QA serious headache there should be a mandatory description/policy how an copr repo containing package(s) we already ship differs from our own package(s).

Let's take for example this http://copr.fedoraproject.org/coprs/davidstrauss/systemd/ an copr repository for an core component of the distribution which has "Description not filled in by author."

Reporter using his repository files bug against systemd now we try to duplicate that and after several ping pongs the reporter finally ( if we are lucky ) utters out I'm not using the official version of systemd I'm using David's version and what do we need to do figure out how David's version differs from our own end result being added steps == added load.

Now you can multiply this if admins get the great ideas of using Fedora's corp as their own personal repository for their infrastructure and all their user base starts filing reports against Fedora and the components we ship in it and all maintainers QA staff and what not start spending significant digging up differences the "Official Package(s)" and "CORP Package(s)"


Recommendation: the %{dist} tag should be overridden in COPR to make it clear that this is a non-Fedora build. Bugs filed against packages with a COPR version in them should be closed immediately as CANTFIX.

So, I don't object to requiring coprs to fill in description... I think that might be a good thing. (adding copr maintainer for comment).

Additionally, might be nice to add a copr faq entry about what to do with bugs in coprs (mail the owner?)

Changing the dist tag is not a great solution, because it means the copr owner must adjust all specs that use dist tag in conditionals. For example, if they just took a Fedora spec and updated it, they would then have to also adjust all the conditionals to get it working.

Otherwise, I don't personally see this as different from any other non Fedora package. People can and do install from source, install from repos.fedorapeople.org, install from pip, gems, etc, etc. When such things are reported as Fedora bugs maintainers figure out that they aren't using the distro provided package and close them.

I do not like idea of requiring description. If I'm creating quick'n'dirty project for myself and I could not submit that form because description is required. I would just put there "blah blah". I would rather change that default text to "Description not filled in by author. Very likely personal repo for testing purpose, which you should not use."

I can very easily change Vendor (but that only shows when you run rpm -qi).

I neither object changing dist tag. I do not think that would need changing spec files. Dist tag should not be used for conditions. But it would require little bit more work.

Replying to [comment:3 kevin]:

So, I don't object to requiring coprs to fill in description... I think that might be a good thing. (adding copr maintainer for comment).

Additionally, might be nice to add a copr faq entry about what to do with bugs in coprs (mail the owner?)

Dont forget about ABRT as well

Changing the dist tag is not a great solution, because it means the copr owner must adjust all specs that use dist tag in conditionals. For example, if they just took a Fedora spec and updated it, they would then have to also adjust all the conditionals to get it working.

And that would just be the price to pay for this stuff unless you want the wild wild rpm west and infra filled with duplicated package or otherwise becoming some form of packaging dumping ground for individual with spec files out of sync with the rest of this stuff.

For example where can I find the cleaning policy for this stuff?

Do we even have ( automated ) process to handle it or is the plan here just to launch this in half baked stated which let's us windup in the exact same scenario we are with the Fedora itself?

And what are we actually trying to solve here with copr that would not be solved by us pushing Alexanders Larsson application/isolation/sandboxin model/proposal forward along with Lennart's "portals" you know putting first back into Fedora as opposed to be solving RHEL and now Centos problems ( which is what the SCL is doing ) but hey that's just me.

Otherwise, I don't personally see this as different from any other non Fedora package. People can and do install from source, install from repos.fedorapeople.org, install from pip, gems, etc, etc. When such things are reported as Fedora bugs maintainers figure out that they aren't using the distro provided package and close them.

The keyword here being "figuring it out" doing so costs time and that time should be on the hands of the indvidual(s) with their own copr certainly not with our maintainers and our services sub community.

For example where can I find the cleaning policy for this stuff?

What do you mean by "cleaning policy"?

Replying to [comment:6 msuchy]:

For example where can I find the cleaning policy for this stuff?

What do you mean by "cleaning policy"?

I thought it was self explanatory in anycase it's just a process that deals with distribution littering which is a real problem we are faced with and costs the community time and money.

Now to something that is obvious to you but less obvious to me what exactly is copr supposed to bring to the table in Fedora and what problems is it solving by doing so because as I see it.

A) If a component can legally be package and shipped in Fedora it should be done so through our official method

B) If a component cant be legally package in Fedora it should not be and it be people redirect to rpmfusion or the likes

C) If we are going to host extra repository's they have to provide real value to the project so the community gets return of investment of community members contributed time or infrastructure resources which means due to A) and B) it can only be application or application stack based repository's?

A sample of repository that does exactly that, as in provide return of investment is the rawhide kernel nodebug repository which allows reporters to run rawhide kernels on GA release provide early feed back to the kernel sub community which delivers it to the upstream kernel, which then get's fixed there and that fix travels back downstream to us.

And a bonus question to fesco and Kevin which also provides insight from infra that spur from this how come something like this ( copr ) can be sanctioned and exist in our infrastructure but when it came to the (server)WG which requires precisely the same infrastructure to succeeds as in separated repository for each application and application stack for/in each "server role" it could not hence...

Was it because I proposed it or because you are to blind to see it's the exact same thing that is requirement ( would cost less infrastructure resource ) just with the added release policy on top of those ( wg's ) repository's?

A) If a component can legally be package and shipped in Fedora it should be done so through our official method

There are two big groups for which this is not an option.

1) Upstream - you are upstream of some SW and you want nightlies and test them. Even before it reach rawhide.

2) Layered applications - Like Spacewalk, which consist of tons of packages and getting them all in Fedora can take ages. E.g. in case of Spacewalk (which I was formerly member of) - after 2 years I was able to get only 1/3 of packages in Fedora.

C) If we are going to host extra repository's they have to provide real value to the project so the community gets return of ...

If you are speaking about those "personal", "quick'n'dirty" or generaly without-description repositories... Well Copr provides place for hacking. And for doing that in repeatable manner and in open space. It can take personal projects out from closet. Which leads to collaboration and to sharing. Which is Copr ultimate goal.

Replying to [comment:8 msuchy]:

A) If a component can legally be package and shipped in Fedora it should be done so through our official method

There are two big groups for which this is not an option.

1) Upstream - you are upstream of some SW and you want nightlies and test them. Even before it reach rawhide.

2) Layered applications - Like Spacewalk, which consist of tons of packages and getting them all in Fedora can take ages. E.g. in case of Spacewalk (which I was formerly member of) - after 2 years I was able to get only 1/3 of packages in Fedora.

This falls under applications or application stacks thus falls under C)

C) If we are going to host extra repository's they have to provide real value to the project so the community gets return of ...

If you are speaking about those "personal", "quick'n'dirty" or generaly without-description repositories... Well Copr provides place for hacking. And for doing that in repeatable manner and in open space. It can take personal projects out from closet. Which leads to collaboration and to sharing. Which is Copr ultimate goal.

Interesting thought process...

Fedora is not the place for such effort in it's current and proposed future state due to it not treating community contributions and projects equally thus limiting the successful of such ideas/projects ( an direction that has always been decided for us as in the community not by us ) significantly so why would I as an individual with a great idea for a project and the will and skill to hack on it chose an environment so hostile to my idea/projects success and on top of that decide to use copr as opposed simply creating an account on github or the likes under real or false identity ( if I would want to remain in the closet ) thus exposing my ideas/project/code to a larger ecosystem and community of coding peers and others to collaborate and share with thus significantly increase my ideas/projects success.

But let's leave that actuality aside and look at a similar goals as in "Fedora as an upstream" we have implemented in the distribution already.

An idea implemented in our distribution in era of more agility,innovation freedom and less github our own fedorahosted.org and proudly sailed under "Start a project and watch it grow" flags all those years.

Wy do you think copr will become less failure then fedorahosted.org?

What's the wow factor it brings to the table so to speak ( wow being like wow I want to join fedora and start using copr )?

Now there is a place for copr like effort for the WG's, application or application stack ( same thing really ) as well as what one of the Dave's threw out there on the kernel channel one time but as an ""personal", "quick'n'dirty" or generaly without-description repository" it just becomes yet another distribution littering resources consuming process in our community as far as I can tell unless the wow factor it brings will blow minds of individuals.

FESCO must have done some researched into copr so there must be some wow factor right?

johan: You seem to be changing the purpose of this ticket to be "shutdown coprs" ?

In any case, for my part:

  • I am in favor of changing the default wording for people who don't fill in description of their copr (as suggested in comment #4). I don't think this needs any fesco involvement particularly.

  • I'm against changing the dist tag for builds, as that doesn't solve the problem and makes things more difficult for maintainers.

  • I'm in favor of coprs and continuing to use/advance them. I think it serves a valuable niche in Fedora. In addition to the use cases msuchy mentioned, another important one is packages in Fedora that cannot be updated in the main repository due to update policies, but that some users may wish to opt-in to updating.

I don't think there's actually any action to take for fesco here, unless we want a vote on 'continue to deploy and use coprs', for which I am a +1.

Replying to [comment:10 kevin]:

  • I am in favor of changing the default wording for people who don't fill in description of their copr (as suggested in comment #4). I don't think this needs any fesco involvement particularly.
    +1

  • I'm against changing the dist tag for builds, as that doesn't solve the problem and makes things more difficult for maintainers.
    We do need to be able to distinguish between Fedora and COPR packages. Right now changing the dist tag seems to be the easiest way; if not that, the best I can think of is changing the bugzilla template to "NEVR and package signature key ID of the affected package" or something similarly ridiculous.

  • I'm in favor of coprs and continuing to use/advance them. I think it serves a valuable niche in Fedora. In addition to the use cases msuchy mentioned, another important one is packages in Fedora that cannot be updated in the main repository due to update policies, but that some users may wish to opt-in to updating.

In some cases, COPRs, are basically a workaround for the Fedora packaging process just being too heavyweight for some cases (the "upstream application stack" is basically this), so heavywheight that it is perceived as not worth doing. We might be fine with that because we perceive a value that the upstreams don't; or perhaps we do need to do something to make our packaging less costly. Food for thought, but also way out of scope for this ticket.

(edited: s/In some sense/In some cases/)

Replying to [comment:10 kevin]:

johan: You seem to be changing the purpose of this ticket to be "shutdown coprs" ?

Is the actuality so frighting that you now accuse me of wanting to shutdown coprs? interesting anyway as I said before it can still be implemented on component level or group of components level as I previously mention but as an method for individuals to deploy their own personal repository I see no benefits.

I don't think there's actually any action to take for fesco here, unless we want a vote on 'continue to deploy and use coprs', for which I am a +1.

Ultimately it will fall on Kevin's and rest of infrastructures hands to ensure that repository's left behind after individuals have left the project is cleaned up but since FESCO members neither shows the will or the intellect to understand this there is no time wasting peoples time trying to improve the distribution and the load on the services sub-communities in the process.

It was as I had expected futile attempt.

I will figure out a way for us to deal with this efficiently in the QA community since FESCO wont hence closing the ticket.

sigh There were two FESCo members already agreeing that something should be done about the descriptions, on the ticket, within 27 hours of the initial filing (not even waiting for a meeting). Is that an insufficient indication of interest in the problem?

And I personally want the dist tag discussion to happen tomorrow. So, reopening. (If you don't want FESCo to consider your proposal any more, feel free to interpret this as me hijacking the ticket because I'm lazy to open another one.)

kevin: "because it means the copr owner must adjust all specs that use dist tag in conditionals"

How many specs actually do that? Naively, I'd guess it wouldn't be too many: referring to the dist tag in a conditional has always seemed terribly fragile to me, so I've always avoided doing it, and I got the impression this was general practice.

I'm with Adam, here. Most specs I've ever seen have used the {{{%{?fedora}}}} (or {{{%{rhel}}}} structure when conditionals are needed.

I think forcing the {{{%{dist}}}} tag to resolve to {{{.fc20copr}}} instead of {{{.fc20}}} would be a nice, easy way to figure out if a package was supported or not.

Frankly, I'm not really sure I give two damns about people using the dist tag in conditionals. http://fedoraproject.org/wiki/Packaging:DistTag implies strongly that using the dist tag in conditionals is a bad idea (this is implied by describing and recommending the use of the much better tags for this purpose).

  • The above mentioned use in conditionals: I did a grep over all package specs in fedora and it looks like this is just 9 packages, so I guess it's pretty small. ;)

  • It doesn't really solve the problem very well. True, some small number of people when reporting bugs might include the rpm -q of the affected package, but in my experence, most people don't. So, you will still have to go back and forth with the reporter to figure that out.

  • abrt shouldn't be a factor here, as reports using it will fail due to non matching buildid on the retrace server. (I think thats the case at least).

Anyhow, if others all feel changing that is worthwhile, I will gracefully accept the decision.

Given that I was planning on working with the baseWG ( and already told that I would do ) and updating all relevant spec files to the package set being used to current guidelines and preferably macro-nize as I could in the process, which would make it a bit more agile to make underlying changes for the base in the future as well as cleaning up any kind 3rd party or downstream conditionals while I was at it.
( since most obviously we are upstream for rhel/epel/centos and what not and as we have to carry distribution patches ourselves against our upstream so should they downstream with them regardless if they are spec files or not ).

Hence I have to ask is there not a cleaner way then to deal with this other than resorting to some form of spec file hacks? ( Having to add those would ofcourse render my efforts effectively utterly and completely useless so I can just as well scrap that off my todo list if that's the case).

AGREED: proposal about adding dist tag didn't pass (+4,-5,0) (mmaslano, 18:44:11)
AGREED: interested parties work with copr maintainer for vendor tag and description changes out of band (+5,-0,0) (mmaslano, 18:52:33)

Login to comment on this ticket.

Metadata