Can we try config_opts['dnf_command'] = "/bin/dnf-4"?

Can we try config_opts['dnf_command'] = "/bin/dnf-4"?

We should try things somewhere first and ship them to production only once they work :/

Untagged

$ koji list-history --build dnf5-5.0.12-1.fc39
Fri May 26 11:02:46 2023 dnf5-5.0.12-1.fc39 tagged into f39-updates-candidate by nsella
Fri May 26 11:02:50 2023 dnf5-5.0.12-1.fc39 tagged into f39-signing-pending by bodhi
Fri May 26 11:04:16 2023 dnf5-5.0.12-1.fc39 untagged from f39-signing-pending by autopen
Fri May 26 11:04:16 2023 dnf5-5.0.12-1.fc39 tagged into f39-updates-testing-pending by autopen
Fri May 26 11:04:23 2023 dnf5-5.0.12-1.fc39 untagged from f39-updates-testing-pending by bodhi
Fri May 26 11:04:23 2023 dnf5-5.0.12-1.fc39 untagged from f39-updates-candidate by bodhi
Fri May 26 11:04:26 2023 dnf5-5.0.12-1.fc39 tagged into f39 by bodhi
Fri May 26 11:04:33 2023 dnf5-5.0.12-1.fc39 tagged into eln by distrobuildsync-eln/jenkins-continuous-infra.apps.ci.centos.org [still active]
Fri May 26 12:06:36 2023 dnf5-5.0.12-1.fc39 untagged from f39 by humaton

$ koji list-history --build=dnf5-5.0.12-1.eln126
Fri May 26 11:34:45 2023 dnf5-5.0.12-1.eln126 tagged into eln-updates-candidate by distrobuildsync-eln/jenkins-continuous-infra.apps.ci.centos.org
Fri May 26 11:34:49 2023 dnf5-5.0.12-1.eln126 tagged into eln-signing-pending by bodhi
Fri May 26 11:36:38 2023 dnf5-5.0.12-1.eln126 untagged from eln-signing-pending by autopen
Fri May 26 11:36:38 2023 dnf5-5.0.12-1.eln126 tagged into eln-updates-testing-pending by autopen
Fri May 26 11:37:24 2023 dnf5-5.0.12-1.eln126 untagged from eln-updates-testing-pending by bodhi
Fri May 26 11:37:27 2023 dnf5-5.0.12-1.eln126 untagged from eln-updates-candidate by bodhi
Fri May 26 11:37:32 2023 dnf5-5.0.12-1.eln126 tagged into eln by bodhi
Fri May 26 12:06:34 2023 dnf5-5.0.12-1.eln126 untagged from eln by humaton

Successfully waited 0:12 for dnf5-5.0.11-3.fc39 to appear in the f39-build repo

I confirm Koji f39 builds work again.

I have also untagged dnf5-5.0.12-1.fc39 from eln, so we should fully good.

We should try things somewhere first and ship them to production only once they work :/

Hi, sorry for the broken builds. We did not predict this issue. Where can koji's build steps be found? I am referring to the call dnf groupinstall which broke the system. The functionality is indeed implemented, but the compatibility alias is missing.

Is it possible to modify mock to not use groupinstall command?

mock has preliminary support for dnf5 but it needs to be configured. See https://github.com/rpm-software-management/mock/issues/894

When you do this update again, please build it in a side tag and try (scratch) building other packages in that side tag -- do not create a bodhi update until it works.

I am not sure whether it will help if there will be not other packages in the side tag.

https://github.com/rpm-software-management/mock/issues/894

As already opened there ^^^, I'm curious if we want to build Rawhide with DNF5,
or keep building with DNF for some time, even though there's DNF5?

I am not sure whether it will help if there will be not other packages in the side tag.

It will verify that builds are generally possible.

The Mock 3.5 doesn't know anything about DNF5. It simply thinks that /bin/dnf is DNF4.

I think there are two basic options for building Rawhide defaulting to dnf5:

1. building it with dnf4 (python variant, /bin/dnf3), can be done with mock-3.5
2. building it with dnf5, we have to update the Mock to version 4.0 first

The first option requires us to set

# tell Mock to use DNF (already in place?)
config_opts["package_manager"] = "dnf"
# tell Mock dnf resides, this is the new thing!
config_opts["dnf_command"] = "/bin/dnf-3"
# install dnf and core plugins correctly into the buildroot
config_opts["dnf_install_command"] = "install python3-dnf dnf-plugins-core"

The problem is that Mock will search for /bin/dnf-3 also on host to
install the bootstrap chroot, so we should provide a dnf-3 -> dnf link on
builders (F37 I guess). We can fix Mock for this particular issue, but such a
fix is not going to be earlier than in v4.1.

The second option is just config_opts["package_manager"] = "dnf5". Mock
4.0 should know the differences between DNF4 and DNF5, and work correctly.

Also note that by default, Mock uses dnf install @group, not the dnf groupinstall variant. This groupinstall is likely configured somewhere in
Koji (but we could simply change chroot_setup_cmd config option).

/usr/bin/dnf-3 is available and by default installed from Fedora 23.

And there actually is config_opts["system_dnf_command"] that would help us to find it.

If the current mock will call dnf-3 binary it can only require dnf-plugins-core and python3-dnf and drop requirement of dnf. This change is compatible with Fedora 22 to 39.

I would suggest that this is tested in staging koji with Fedora release engineering and the koji and mock developers. Koji does specific changes to the mock configs that used in any build and may need to have changes done in the koji code versus just mock.

I also expect that various other tooling may use hardcoded versions of dnf in places which will need to be worked out also. Places I would expect this to be would be the container building tools, and the general compose tools surrounding pungi.

Ad option 1.

OK, can be tested with Copr config (consists of three repositories, but this
basically mimics what will be part of Rawhide soon, and what DNF team promised
to provide for testing in a side-tag):

$ copr mock-config rpmsoftwaremanagement/dnf5-unstable fedora-rawhide-x86_64 > /tmp/future-rawhide.cfg
$ echo 'config_opts["dnf_command"] = "/usr/bin/dnf-3"' >> /tmp/future-rawhide.cfg
$ echo 'config_opts["dnf_install_command"] = "install python3-dnf dnf-plugins-core"' >> /tmp/future-rawhide.cfg
$ mock -r /tmp/future-rawhide.cfg <some.rpm>

The fact that /bin/dnf-3 is provided by python3-dnf since ever helps us a lot, no additional fixes seem to be needed even with Mock 3.5.

For mock config changes, we need that to happen in koji. We can make mock config changes on builders, but builders have to be able to build for epel7/8/9/fedora37/38/rawhide/eln... we can't make some incompatible change there that breaks all the rest and gets rawhide working.

For the yum to dnf change, koji added a tag config option 'package_manager'.
ie,

 ✗ koji taginfo f39-build
...
  mock.package_manager : 'dnf' [f39]
...

So, perhaps it would make sense to reuse this and just get koji/mock able to use 'dnf5' there and then we can just set it on the f39+ tags when we tag it back in?

CC: @tkopecek @mikem

I don't think untagging dnf5-5.0.12-1.fc39 alone is sufficient. That build appears to go hand in hand with dnf-4.16.0-1.fc39, because /etc/dnf/dnf.conf moved from dnf-data (subpackage of dnf) to libdnf5 (subpackage of dnf5). dnf-4.16.0-1.fc39 is still tagged. That results in the registry.fedoraproject.org/fedora:39 container image having no package owning that file. The file still exists because something writes tsflags=nodocs there during the build process, but that doesn't appear to be a valid configuration without a [main] section header.

[carl@teal:~]$ podman run --rm registry.fedoraproject.org/fedora:39 dnf repolist
Config error: Parsing file "/etc/dnf/dnf.conf" failed: Parsing file '/etc/dnf/dnf.conf' failed: IniParser: Missing section header at line 1

I'll untag that too...

My apology for the problem. We cooperate with @praiskup to resolve the issue, The next release will be created in the side-taq.

@kevin using mock.package_manager=dnf5 should be straightforward. It directly translates to mock's option.

mock.package_manager=dnf5 should be straightforward

Hopefully yes, but then Mock 4.0 is required on builders (and a very fresh code is used).

According to the chat with DNF folks, "option 1" is still the prefered thing
(continue with building Rawhide with DNF4 for some time). And after some
time switch to "option 2" (build rawhide with DNF5).

From Mock perspective, this two-phase process is reasonable too - we'd have some time to test
and fix the fresh DNF5 support, and distribute Mock v4.x to a wider audience.

Hi, seems that we are hitting from today failures related to this in Anaconda tests (Rawhide).

 dnf install -y 'dnf-command(copr)'
Config error: Parsing file "/etc/dnf/dnf.conf" failed: Parsing file '/etc/dnf/dnf.conf' failed: IniParser: Missing section header at line 1

We don't mess with dnf.conf at all. Should be original one from the Fedora Rawhide container.

https://github.com/rhinstaller/anaconda/actions/runs/5107331516/jobs/9187025071

Todays rawhide container should have the older dnf and work again, if not, let us know.

Seems to be working good for us. Thanks!

Since this issue was about untagging, I created a new issue #11446 related to
the configuration changes needed in Koji.