Given the recent synchronization of Python and Fedora schedules, we will most likely be usually doing this right after branching. This is both bad and good:

• right after branching, there are no rawhide modules to query (until they are rebuilt)
• the rawhide modules need to build in order to be created - if this happens after our side tag is merged, we don't need to track it - either they build with new Python or they don't exist

However, I would rather not use this as a long term strategy.

We have updated Python in rawhide to 3.9. I'd like to rebuild all rawhide modular streams that need it. I don't know what to do. Any advice?

Based on information from #165 I was able to identify the following modular packages that need to be rebuilt:

$ repoquery --setopt=*.module_hotfixes=yes --repo=rawhide-modular --whatrequires 'python(abi) = 3.8'  --whatrequires 'libpython3.8.so.1.0()(64bit)'
python3-aexpect-0:1.5.1-6.module_f33+8267+d222b029.noarch
python3-avocado-0:79.0-1.module_f33+8897+4c9a3c59.noarch
python3-avocado-plugins-glib-0:79.0-1.module_f33+8897+4c9a3c59.noarch
python3-avocado-plugins-golang-0:79.0-1.module_f33+8897+4c9a3c59.noarch
python3-avocado-plugins-loader-yaml-0:79.0-1.module_f33+8897+4c9a3c59.noarch
python3-avocado-plugins-output-html-0:79.0-1.module_f33+8897+4c9a3c59.noarch
python3-avocado-plugins-result-upload-0:79.0-1.module_f33+8897+4c9a3c59.noarch
python3-avocado-plugins-resultsdb-0:79.0-1.module_f33+8897+4c9a3c59.noarch
python3-avocado-plugins-varianter-cit-0:79.0-1.module_f33+8897+4c9a3c59.noarch
python3-avocado-plugins-varianter-pict-0:79.0-1.module_f33+8897+4c9a3c59.noarch
python3-avocado-plugins-varianter-yaml-to-mux-0:79.0-1.module_f33+8897+4c9a3c59.noarch
python3-avocado-vt-0:77.0-2.module_f33+9012+f22a9456.noarch
python3-librealsense-devel-0:2.33.1-4.module_f33+8623+c2e7b150.x86_64

$ repoquery --setopt=*.module_hotfixes=yes --repo=rawhide-modular --whatrequires 'python(abi) = 3.8' --whatrequires 'libpython3.8.so.1.0()(64bit)' --source
avocado-vt-77.0-2.module_f33+9012+f22a9456.src.rpm
librealsense-2.33.1-4.module_f33+8623+c2e7b150.src.rpm
python-aexpect-1.5.1-6.module_f33+8267+d222b029.src.rpm
python-avocado-79.0-1.module_f33+8897+4c9a3c59.src.rpm

I have no idea how to proceed further.

We will be doing mass rebuild for Python 3.10 in ~1 month:

https://fedoraproject.org/wiki/Changes/Python3.10

Since there has been no new information here, we plan to ignore modules entirely. Let me know if we should do something else.

Python 3.10 rebuild is over. Nothing was done with modules.

I can see you can discover affected modular packages now. The next step is mapping the package names to modules. Use "dnf module provides RPM_PACKAGE" for that. Finally rebuilding a module is a matter of pushing an empty git commit to modules name space and invoking "fedpkg module-build --optional rebuild_strategy=all" in the order of modular dependencies.

Thanks Petr!

I can see you can discover affected modular packages now.

Is that discovery complete thou? Are there no modules that are hidden from the users but available to build other modules?

The next step is mapping the package names to modules. Use "dnf module provides RPM_PACKAGE" for that.

For reference, this is an example invocation.

$ dnf --repo=rawhide-modular module provides python3-avocado-plugins-resultsdb-0:88.1-1.module_f35+12047+98db6e8d.noarch
python3-avocado-plugins-resultsdb-88.1-1.module_f35+12047+98db6e8d.noarch
Module   : avocado:latest:3520210518132924:f27b74a8:x86_64
Profiles : 
Repo     : rawhide-modular
Summary  : Framework with tools and libraries for Automated Testing

I suppose the modelar repo and branch is avocado and latest, correct?

Finally rebuilding a module is a matter of pushing an empty git commit to modules name space and invoking "fedpkg module-build --optional rebuild_strategy=all" in the order of modular dependencies.

Would that not build the module for other Fedora releases and EPELs? Can we build it in rawhide only?

Are there no modules that are hidden from the users but available to build other modules?

It's complete for module builds in a repository. Modules built but not pushed into a repository cannot be searched with DNF. DNF works only with repositories. So the answer is no, this is not a complete discovery.

In general, all finished rawhide module builds go into a repository. Exceptions are modules maching *-bootstrap glob and few others which relengs omit intentionally. There was a discussion whether the omitted ones should go to a dedicated repository but that did not lead to any conclusion. You can use https://mbs.fedoraproject.org/module-build-service/2/module-builds/?base_module_br_stream=f35&state=5&short=true&per_page=100 HTTP query to list all finished non-scratch module builds for f35 platform. Though there is no direct way of introspecting their packages. You can use ODCS to compose them and then search the composes with DNF.

For reference, this is an example invocation.

$ dnf --repo=rawhide-modular module provides python3-avocado-plugins-resultsdb-0:88.1-1.module_f35+12047+98db6e8d.noarch
python3-avocado-plugins-resultsdb-88.1-1.module_f35+12047+98db6e8d.noarch
Module : avocado:latest:3520210518132924:f27b74a8:x86_64

This means "avocado" module name, "latest" module stream". I.e. "latest" branch of "modules/avocado" dist-git repository. See https://src.fedoraproject.org/modules/avocado/tree/latest.

Would that not build the module for other Fedora releases and EPELs? Can we build it in rawhide only?

Good catch. Yes, it would build them for all platforms listed or stream-expended in their source (modulemd YAML) files. Since the builds would not go into Bodhi, the unneeded builds would not bother users. Only Koji.

If is an issue for you, I believe you can override the build-required platform stream with "fedpkg module-build --buildrequires platform:f35" command. That could only constrain builds to Fedora 35.

But I have no idea what would happen later when a regular build for all platforms happened if one of the build-required dependencies had been rebuilt for Rawhide only. (E.g. you enforce the platform when building perl-bootstrap:5.32 and later, a packager attempts to build perl:5.32 which happens to build-require perl-bootstrap:5.32.) My latest experience is that MBS would stop building perl:5.32 for non-rawhide. But my latest experience is from testing a new v3 modulemd-packager format which behaves differently by definition (no stream expansion, static context values). So take my warning with a pinch of salt.

Here https://pagure.io/releng/issue/8616 was omitting the bootstrap modules implemented. Current Pungi configuration https://pagure.io/pungi-fedora/blob/main/f/fedora.conf lists only "rpm" and "perl*bootstrap" modules.

Since perl-boostrap does not use Python and "rpm" was never built for f35, I think you can skip checking non-repository modules for now.