#2391 roles/bodhi2: Set EPEL to use zstd repodata and disable unusable features
Opened 3 months ago by ngompa. Modified a month ago
fedora-infra/ ngompa/ansible zstd-epelrepodata  into  main

@@ -26,24 +26,38 @@ 

  # Each section only inherits from DEFAULT, so `release` will not inherit from `prefix`.

  # 'release' take precedence over 'prefix' as it is more specific.

  [prefix.FEDORA-EPEL]

- compatibility = True

- updateinfo-compress-type = BZ2

- drpms_enabled = True

- general-compress = False

- # The following should be superflous as they are overridden by compatibility flag

- repodata-compress-type = gz

+ compatibility = False

+ updateinfo-compress-type = ZSTD

+ drpms_enabled = False

+ general-compress = True

+ repodata-compress-type = zstd

  zchunk = False

- sqlite_enabled = True

+ sqlite_enabled = False

  

  [prefix.FEDORA-EPEL-NEXT]

- compatibility = True

+ compatibility = False

+ updateinfo-compress-type = ZSTD

+ drpms_enabled = False

+ general-compress = True

+ repodata-compress-type = zstd

+ zchunk = False

+ sqlite_enabled = False

+ 

+ [release.EPEL-9]

  updateinfo-compress-type = BZ2

- drpms_enabled = True

- general-compress = False

- # The following should be superflous as they are overridden by compatibility flag

- repodata-compress-type = gz

+ repodata-compress-type = xz

+ general-compress = True

  zchunk = False

- sqlite_enabled = True

+ drpms_enabled = False

+ sqlite_enabled = False

+ 

+ [release.EPEL-9N]

+ updateinfo-compress-type = BZ2

+ repodata-compress-type = xz

+ general-compress = True

+ zchunk = False

+ drpms_enabled = False

+ sqlite_enabled = False

  

  [release.EPEL-8]

  updateinfo-compress-type = BZ2

Zstd repodata support was introduced in DNF quite some time ago, and
is quite a bit faster and less resource intensive.

Additionally, there is no point in producing sqlmd or DeltaRPMs when
support for both has been removed from RHEL for a fair bit of time.

Regardless, we should leave things mostly as-is for EPEL 9, and thus
we retain older xz compression settings for that.

Signed-off-by: Neal Gompa ngompa@fedoraproject.org

I think we should just drop the [prefix.FEDORA-EPEL-NEXT] section. We're not creating any more and the only one still active is EPEL-9N, so we can just set the the options for that one. While we're at it we can drop the [release.EPEL-8N] section since it's already retired.

If I'm reading this right, this will be changing EPEL-9 from compatibility = True to compatibility = False. If we want to keep that the same we'll need an override in this section.

If you want to retain settings for EPEL-9 / EPEL-9N just copy the existing lines of prefix.FEDORA-EPEL section in two release.EPEL-9 / release.EPEL-9N sections, then changes in prefix.FEDORA-EPEL will apply only to EPEL-10.

Note that AFAIK the compatibility=True flag will override the following settings to:

repodata-compress-type = gz
zchunk = False
sqlite_enabled = True

You can only change them if compatibility=False.

I turned compatibility to False because shipping sqlmd is pointless for EPEL 9 since nothing supports it.

rebased onto 1bb2581

2 months ago

So, to summarize, this:

  • drops sqlite_enabled for epel9 / epel9-next

  • switches epel10 to zstd

I think dropping sqlite might be fine for epel9/epel9-next.

Note that switching epel10 to zstd means rhel8/9 hosts can't mergerepo_c / createrepo_c those repos because it doesn't understand zstd).
( https://issues.redhat.com/browse/RHEL-67689 )
So that might be something to run by the EPEL sig?

Fedora infrastructure doesn't run repository generation on RHEL, does it?

No, but there's external folks who do... possibly centos?

CBS runs on RHEL 9 machines I believe. I guess that means we need a patched createrepo for CBS infrastructure?

FYI: @arrfab

I confirm that builders behind cbs.centos.org are all migrated to RHEL9 and so are using createrepo_c-0.20.1-2.el9.x86_64

Yeah, rhel8/9 createrepo_c doesn't have zstd support (yet).

https://issues.redhat.com/browse/RHEL-67689

There is rhel/stream createrepo_c with zstd enabled at https://copr.fedorainfracloud.org/coprs/amatej/createrepo_c/

Currently Fedora 41 updates and now rawhide are using zstd.

Ideally we would wait for rhel8/9/stream8/9 to update ctreaterepo_c to support it before moving epel... but if thats going to be a long time or never, perhaps we don't wait.

Metadata