I'm not entirely sure I understand this properly, so here are the questions I think I know the answers to, but would like to get the real answers explicitly.

1. As a user of Fedora 30, when I install e.g. ripgrep, where do I get it from? Is it a module generated from rawhide? Or just at branching inherited build from rawhide? I see you talk about a module yet you also say why you don't do a module.

2. If there is a CVE found in a crate that affects such package, how do I get the update?

3. Is releng entirely sure what do you need? It doesn't appear so from their answers.

I'm not entirely sure I understand this properly, so here are the questions I think I know the answers to, but would like to get the real answers explicitly.

As a user of Fedora 30, when I install e.g. ripgrep, where do I get it from? Is it a module generated from rawhide? Or just at branching inherited build from rawhide? I see you talk about a module yet you also say why you don't do a module.

As far I understood, it comes from a module and it will built against that particular release platform.

If there is a CVE found in a crate that affects such package, how do I get the update?

I think you will get it as a module update.

Is releng entirely sure what do you need? It doesn't appear so from their answers.

I think I am sorta sure what he needs, but I need to think about it a bit more, probably I am missing something.

Also, I think we need to weigh in modularity folks as well, not sure if its going to break anything, but you never know with modularity.

@ignatenkobrain Please correct me if I am wrong in the above answers.

1. You get ripgrep from a module, but it doesn't use crates from any repo, it builds them during build. For any version of Fedora. Doing a module with an application is what we do and plan to do. Doing a module with crates (essentially devel packages) is not feasible at this time due to inefficiency of modularity (we might do it at some point in future).

2. For an application you get it from a module which doesn't use any crates from Fedora RPM repos, it uses distgit to build all the dependencies.

Here you can find modulemd for ripgrep: https://src.fedoraproject.org/modules/ripgrep/blob/latest/f/ripgrep.yaml

We would be fine even not building crates at all, but then we would loose all nice things we get now: release-monitoring, koschei.

(I'm inclined to +1 this, but I want to wait before others ask clever questions I haven't thought about yet.)

I am +1 to allow them to opt out of branching.

Nevertheless, IMHO it is bad that Rawhide is abused to build packages that are not meant to be shipped to users (if I understood correctly that this is happening).

We would be fine even not building crates at all, but then we would loose all nice things we get now: release-monitoring, koschei.

I suppose this means there is no release-monitoring and koschei support for modules? Is someone working on this/is it tracked?

I suppose this means there is no release-monitoring and koschei support for modules? Is someone working on this/is it tracked?

Currently I don't have any plans to have Koschei for modules. Koschei relies on scratch builds, but there are no scratch builds for modules implemented in MBS yet. The MBS feature is tracked in fm-orchestrator ticket 752.

it is bad that Rawhide is abused to build packages that are not meant to be shipped to users

I agree and after seeing this ticket being filed I talked with @ignatenkobrain and we agreed to a plan/proposal that, once implemented, would put an end to this abuse. After consultation with some other package maintainers we will send more details to mailing lists.

FTR, here is our proposal: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/ZDG6EX3ZDZZXEZU5PTGIOLMV4U7ATF2R/

I suppose that would take time to implement so the original request still stands, correct?

I suppose that would take time to implement so the original request still stands, correct?

Yes.

+1 to allow opt out from branching

Rust SIG is not updating crates in released Fedora because it is painful (you
need to add new packages, create tons of overrides and sometimes you can't
update it because it is incompatible). Those packages are pretty much not
maintained but stay in the repo. After we do compiler upgrade (we do that for
all Fedora and EPEL), some packages become FTBFS there because new compiler
might convert some warning to an error. No new packages added to released Fedora
either.

Rust crates are mostly used when building app and we just generate modulemd
file based on rawhide metadata including all crates there. ...

So WRT crates on stable fedoras/epel we
- update compiler to the latest greatest version
- we never update build dependencies
- but still we are supposed to be able to support/build the "app"

Those facts aren't aligned according to the same goal IMO. Any
supported app needs to be re-buildable even on the stable fedora (unless
you support Rawhide only), and thus you should either not upgrade
compiler, or update build dependencies. Otherwise, transitively, you are
unable to update the app for security fixes (especially if this is all about
bundling).

It also sort of looks like the build deps are in fedora only because of
single one application... is this the specific problem of your use-case,
compared the any other '*-devel' package in Fedora? Note that there are
certain security promises behind adding package to Fedora, so orphaning
them after branching is not really what we should do.

Or can you please elaborate on this problem?

Q: Why not to create module with all crates and build it everywhere?
A: With current modulemd format and MBS/Koji it would take ages to perform any
update to > package. For instance, ripgrep module has only 19 levels of
building (that buildorder parameter) and it takes 4+ hours to build all that.
If we would describe full rust ecosystem (500+) packages, we would need to
rebuild it all each time some update happens (because there is no
before/after, see https://github.com/fedora-modularity/libmodulemd/issues/96)
and it would take few days probably.

This doesn't seem to be resolved FAQ.

Can anyone from modularity comment on this, and say what could be done
better, and when it can be done? And how long it would take to implement?
I mean, I'm unable to say whether we should +1 this proposal, because it's
really out of scope for modularity...

Otherwise to me this use-case seems to be exactly what modularity is
aimed to be about (with e.g. two modules, say 'crates-devel' module being
a build-only set of packages for the second module named 'crates'?). Parametric
chain rebuilds. The fact that it doesn't scale now is matter of quality of
implementation, and temporary problem, or? @psabata @langdon ?

Those facts aren't aligned according to the same goal IMO. Any
supported app needs to be re-buildable even on the stable fedora (unless
you support Rawhide only), and thus you should either not upgrade
compiler, or update build dependencies. Otherwise, transitively, you are
unable to update the app for security fixes (especially if this is all about
bundling).

They are, because module with the app contains all crates.

It also sort of looks like the build deps are in fedora only because of
single one application... is this the specific problem of your use-case,
compared the any other '*-devel' package in Fedora? Note that there are
certain security promises behind adding package to Fedora, so orphaning
them after branching is not really what we should do.

That is a problem of java and golang too.

Otherwise to me this use-case seems to be exactly what modularity is
aimed to be about (with e.g. two modules, say 'crates-devel' module being
a build-only set of packages for the second module named 'crates'?). Parametric
chain rebuilds. The fact that it doesn't scale now is matter of quality of
implementation, and temporary problem, or? @psabata @langdon ?

I am fine with having rust-ecosystem module which contains all crates, but it will take extreme amount of time to build due to inefficiencies in modulemd format/MBS/Koji/whatsoever.

I am fine with having rust-ecosystem module which contains all crates, but it will take extreme amount of time to build due to inefficiencies in modulemd format/MBS/Koji/whatsoever.

Ok, it means that https://github.com/fedora-modularity/libmodulemd/issues/96 solution will solve this issue too, and that work-around by relaxing policy isn't necessary. Or if anything, it should be a temporary work-around, right?

Ok, it means that https://github.com/fedora-modularity/libmodulemd/issues/96 solution will solve this issue too, and that work-around by relaxing policy isn't necessary. Or if anything, it should be a temporary work-around, right?

It has to be implemented in MBS too. And deployed in infrastructure ;)

It is temporary workaround, in future we won't build crates even in rawhide, we will use separate koji (possibly, see MBI thread on devel@) and then just ship all crates in one module in Fedora.

If we do this, how would security updates be supplied to stable Fedora releases?

If we do this, how would security updates be supplied to stable Fedora releases?

Security updates of what? If you are talking about crates, they are not shipped to users at all. If you are talking about applications, we will build new versions of modules.

On Thu, 2019-02-07 at 16:44 +0000, Igor Gnatenko wrote:

Security updates of what? If you are talking about crates, they are
not shipped to users at all. If you are talking about applications,
we will build new versions of modules.

Yeah I meant applications. So you would build new versions of the
modules, which would only come from the master branch, and then you
would ship those to the stable Fedoras? If so, +1 from me. Just want to
make sure we have a plan for continuing to update stable releases.

Not from master branch, but yes ;)

One more thing.

1. I install rawhide and install crates from the Fedora repo
2. I stay on f30 after branching
3. how do I get notified that the crates I've installed will get no more (security) updates?

I'd probably appreciate an example of crate library, and the app - what happens before and after branching :-). What packages will be opted-out then?

Not from master branch, but yes ;)

.. because I thought that we branch only master.

I'd probably appreciate an example of crate library, and the app - what happens before and after branching :-). What packages will be opted-out then?

So we have rust-grep (crate) and rust-ripgrep (app).

• crate should have only master branch and (for now) build only in rawhide (in future would be built in MBI)
• app should have only latest branch and always be built as a module (which includes building crate)

Aah, so it means that neither of rust-grep and rust-ripgrep pair are expected to be available in stable fedoras, i.e. fedora-29 or fedora-30 (once we branch it from Rawhide), right?
It sort of makes sense if you have "Rawhide-only" users.

But... It seems to me that you don't actually want to land this software into Fedora which in turn means that it probably is an usecase for copr; but at the same time you claim Copr doesn't match your needs. I'd say that it is absolutely necessary to precisely define problems with Copr and to make you comfortable with using it.

Aah, so it means that neither of rust-grep and rust-ripgrep pair are expected to be available in stable fedoras, i.e. fedora-29 or fedora-30 (once we branch it from Rawhide), right?
It sort of makes sense if you have "Rawhide-only" users.

rust-ripgrep will be available through module in all supported Fedora.

But... It seems to me that you don't actually want to land this software into Fedora which in turn means that it probably is an usecase for copr; but at the same time you claim Copr doesn't match your needs. I'd say that it is absolutely necessary to precisely define problems with Copr and to make you comfortable with using it.

Please see MBI document for the reasons "Why not COPR".

rust-ripgrep will be available through module in all supported Fedora.

I must be missing something; if rust-ripgrep depends (even if build-time only) on rust-grep - and rust-grep wont be branched (master only) and won't be in module, how do you plan to update rust-grep in stable fedora (e.g. for security issues)?

Please see MBI document for the reasons "Why not COPR".

I've seen that, but there's a lot of questions for me ... it would be really awesome to have a document saying "why we can't fix copr".

AGREED: Opt-out from branching for rust packages is approved (+6,0,-0) (jforbes, 15:37:25)