I ended up enabling the eog module on my system (f29 upgraded from f28) and getting such a broken system because of:

$ sudo dnf update

Problem 1: cannot install the best update candidate for package bubblewrap-0.3.0-2.fc29.x86_64
- package bubblewrap-0.3.0-2.module_2123+73a9ef6f.x86_64 is disabled
Problem 2: cannot install the best update candidate for package eog-3.28.3-1.fc29.x86_64
- package eog-3.28.3-1.module_2123+73a9ef6f.x86_64 is disabled

which could be seen as a suggestion from dnf to enable this module ;)

Adding @otaylor

We need to figure out if there's a way we can identify these special flatpak modules and suppress them from the standard module repository. Can you describe a bit about how they're used in the build process, please?

@teuf note that's https://bugzilla.redhat.com/show_bug.cgi?id=1616118 . You're right that it's particularly unfortunate in conjunction with this problem, as it exposes these modules to users in a confusing 'error message'.

There isn't a criterion this obviously fits under (though we could perhaps bend the "appropriately install software" one a bit), but on the merits, I agree with @walters that this is a significant problem and we absolutely should get it fixed ahead of Final.

@walters , Other than there being files in the /app hierarchy unexpectedly, what "brokenness" do you see here? I mean, it's clearly wrong, but that doesn't necessarily equate to "broken".

@walters , Other than there being files in the /app hierarchy unexpectedly, what "brokenness" do you see here? I mean, it's clearly wrong, but that doesn't necessarily equate to "broken".

The brokenness comes because the packages from the runtime module have the same name as the standard packages, and dnf might choose to install them instead of the ones that are needed - the ones that provide files in /usr.

These modules are used to build Flatpak containers via and that is the means of distribution of the content. You can always identify a flatpak module because it depends on the flatpak-runtime module (directly, or potentially, indirectly - though we could forbid that.)

• Do we want them tagged somewhere? (A f29-flatpak-modules tag, say.) If they aren't tagged anywhere, I believe the koji build, and hence the RPMs/SRPMs will be garbage collected. We still have the build information and the sources in src.fedoraproject.org.

• Is the dependency on flatpak-runtime sufficient for special handing, or do we want something in the xmd?

@walters , Other than there being files in the /app hierarchy unexpectedly, what "brokenness" do you see here? I mean, it's clearly wrong, but that doesn't necessarily equate to "broken".

The brokenness comes because the packages from the runtime module have the same name as the standard packages, and dnf might choose to install them instead of the ones that are needed - the ones that provide files in /usr.

OK, yeah that's a problem.

These modules are used to build Flatpak containers via and that is the means of distribution of the content. You can always identify a flatpak module because it depends on the flatpak-runtime module (directly, or potentially, indirectly - though we could forbid that.)

Do we want them tagged somewhere? (A f29-flatpak-modules tag, say.) If they aren't tagged anywhere, I believe the koji build, and hence the RPMs/SRPMs will be garbage collected. We still have the build information and the sources in src.fedoraproject.org.

Is the dependency on flatpak-runtime sufficient for special handing, or do we want something in the xmd?

If we support the indirect dependency on flatpak-runtime, then I think we do need to have something added to XMD that we can use to determine that this is a flatpak. If we forbid the indirect dep, then we can just use the dependency information.

Either way, this is going to mean modifying the tools that do the tagging to not tag them as fXX-modular-candidate.

Also, there are questions around Bodhi and workflow - if we decide that we need the builds to not be garbage collected:

• Do we require packagers to do Bodhi updates for Flatpak modules after bodhi activation to get them into the tag? Do we add something to bodhi's Flatpak support so that the module tagging follows the container tagging?
  • The flatpak-common module I'm working on adds another wrinkle to this that makes my head hurt - as soon as you build flatpak-common, subsequent Flatpak container builds will pull packages from that build, so if we're saving the builds for history, that flatpak-common build needs to be tagged. This gets super-complicated!

And are there going to be other non-Flatpak modules like this - that are container-only and we want to exclude from the modular compose? If so, it might make more sense to exclude them at compose time, and not have to create separate tags, bodhi releases, etc.

I guess https://pagure.io/fm-orchestrator/issue/974 should solve this issue, but I am not sure about the priority on this.

If its on hold, is there anyway that we can block them before going into the compose? I guess that needs some tooling to identify them and block them before going into the compose.

cc'ing @jkaluza @lsedlar if we can do any filtering of packages in pungi configs.

Do we require packagers to do Bodhi updates for Flatpak modules after bodhi activation to get them into the tag? Do we add something to bodhi's Flatpak support so that the module tagging follows the container tagging?

I guess thats the plan, @bowlofeggs can you confirm?

I don't think https://pagure.io/fm-orchestrator/issue/974 has anything to do with this. Flatpak modules will (indirectly) require platform:f29.

Solutions I can think of here:

1. Give Flatpaks modules their own tag in Koji - this would require fedpkg changes to know to build modules that are identified as flatpaks in that tag (either by the presence of a container.yaml or because we've moved flatpaks to their own namespace in dist-git)
2. Add some very special compose option to pungi so you can say "include all modules in this f29-modular, except for those that depend on flatpak-runtime".

Note that for f29, since we're past bodhi-activation, we just need to untag eog and flaptak-runtime builds, if that hasn't been done already.

@mohanboddu I would probably not advocate putting the flatpak modules into Bodhi, since we don't intend to ship the modules directly. At least, that's my understanding - the modules are used to build the flatpak (and I guess are included inside the flatpak), but we don't want to ship them as bare module repositories as I understand it. Bodhi is a control system for artifacts we want to ship (well, not all artifacts, as it doesn't control when we ship ISOs ☺).

I'd probably advocate for @otaylor's solution #1 in his comment immediately prior to mine.

So people stop hitting this, can someone as a short term solution move all the 'eog' and 'flatpak-runtime' modules from f29-modular-updates back to f29-modular-updates-candidate? I don't have permission:

$ koji move-build f29-modular  f29-modular-updates-candidate \
     eog-master-20180821163756.775baa8e 
ActionNotAllowed: tag requires autosign permission

(I don't want to leave the builds entirely untagged, since I think they might be GC'ed then.)

I can untag that one, but can you provide a list (or way to detect) all the flatpaks there so we can make sure and untag them all?

Closing the ticket, please reopen if anything else needed.