Old major versions of EPEL such as EPEL 6 are available in the Fedora archive, and have repos set up in MirrorManager to point users who request them to mirrors of the Fedora archive. It would be beneficial for some EPEL users, including users of RHEL EUS, to also get old minor versions the same way. While EPEL doesn't currently have minor versions, snapshots that emulate the minor versions (i.e. only contain packages that were compatible with a RHEL minor version) do already exist in the Fedora archive.
I would like for MirrorManager to respond with minor version paths in the mirrored archives if a user requests an EPEL minor version. For example:
epel-8.8
/pub/archive/epel/8.8/
epel-9.2
/pub/archive/epel/9.2/
The list of minor versions we have EPEL snapshots for can be viewed here. Ideally we would set up MirrorManager responses for each of these.
No deadline, this is a nice to have that isn't currently blocking anything. I will note that the status quo has an load impact on the main archive.fpo server(s). Currently when users want to set this up, the most common course of action taken is directly configuring their systems to use a baseurl pointing at archive.fpo. Having the EPEL minor versions in MirrorManager will spread that load to the mirror network. It will also remove the need for users of RHEL EUS to manually configure their baseurl this way.
Metadata Update from @adrian: - Issue assigned to adrian
I will take a look at it.
It has come to my attention that this was suggested previously, and there were some mixed feelings about it. I'm going to stick it on the agenda for the next EPEL Steering Committee meeting (two days from now) to make sure the committee members have a chance to voice their concerns before we move forward. Please put this on hold until then.
Metadata Update from @phsmoura: - Issue priority set to: Waiting on Assignee (was: Needs Review) - Issue tagged with: low-gain, low-trouble, ops
I brought this up at the committee meeting today. No one seemed opposed to it, but since I didn't file an issue until the day of the meeting, the consensus was to leave it open for a week to allow time for comments, and then vote on it next week.
https://pagure.io/epel/issue/278
Today the committee approved implementing this. We can proceed with adding the minor versions we have snapshots for (everything except 8.0, 8.6, and 9.0).
@carlwgeorge
I created one repository as a test. Please try:
curl -s "https://mirrors.fedoraproject.org/metalink?repo=epel-8.8&arch=x86_64&country=global"
Following steps where necessary on the database level. First figure out the ID of the directory:
=> select id,name from directory where name like 'pub/archive/epel/8.8.2023-11-14/Everything/x86_64/repodata'; 668521 | pub/archive/epel/8.8.2023-11-14/Everything/x86_64
then get the product ID:
=> select * from product; id | name | publiclist ----+--------+------------ 1 | EPEL | t
The create a new version:
=> insert into version (name, product_id, is_test, display, sortorder) values ('8.8', 1, false,false, 0); => select * from version where name = '8.8'; 934244 | 8.8 | 1 | f | f | | t | 0 |
With all that information it should now be possible to create the repository:
=> insert into repository (name, prefix, category_id, version_id, arch_id, directory_id , disabled) values ('pub/archive/epel/8.8.2023-11-14/Everything/x86_64', 'epel-8.8', 4, 934244, 3, 668521, false);
I can confirm that curl works as expected. My only question is, should we be using the date directories like 8.8.2023-11-14 instead of just 8.8?
I would say no, the date ones are there to indicate when the archive was made, for mm, the minor release should be just fine...
I've created the repos in Mirrormanager, they all seem to work except 8.3, 8.4, and 8.5. I'm going to investigate why, but probably tomorrow.
For posterity, here is the error I'm seeing with those three releases.
❯ curl 'https://mirrors.fedoraproject.org/metalink?repo=epel-8.4&arch=x86_64' <?xml version="1.0" encoding="utf-8"?> <metalink version="3.0" xmlns="http://www.metalinker.org/" type="dynamic" pubdate="Mon, 15 Jul 2024 17:36:43 GMT" generator="mirrormanager" xmlns:mm0="http://fedorahosted.org/mirrormanager"> <!-- pub/archive/epel/8.4.2021-11-30/Everything/x86_64/repodata/repomd.xml not found or has not metalink --> </metalink>
Any news here on those last archives?
We also need to add the recently archived 9.4 as well.
https://dl.fedoraproject.org/pub/archive/epel/9.4-2024-11-12/
This just came up over in the discussion forum.
https://discussion.fedoraproject.org/t/two-rhel-9-4-systems-one-produces-404-errors-for-epel-repositories/139130
Hey! I looked back at this issue, and I think there are two issues: - the 9.4 directory is not in /srv/pub/archive/fullfiletimelist-archive - the pattern "pub/archive/epel/[\\d\\.-]+/Everything/.*" is not in the scan-primary-mirror.toml (that's something I can fix)
/srv/pub/archive/fullfiletimelist-archive
"pub/archive/epel/[\\d\\.-]+/Everything/.*"
scan-primary-mirror.toml
Also, those directories are currently in the EPEL category in MirrorManager. Should they be put in the Fedora Archive category?
EPEL
Fedora Archive
Updated the archive fullfiletimelist... ( by running: time sudo -u ftpsync /usr/local/bin/update-fullfiletimelist -l /pub/fedora-secondary/update-fullfiletimelist.lock -t /pub archive on bodhi-backend01)
And yeah, they should probibly be in Fedora Archive.
The EPEL 9.x repositories are created manually in the database. We do not try to detect anything in archive/ to not conflict with something.
archive/
The idea was that once the files have been detected and are added to the database the repository needs to be added manually using SQL. At least that is how I remember the discussion. It was a 9.x only thing that is why we thought there is no need to automate it. Especially as we do not want any MirrorManager repositories created from archive content.
I tried to add it to the database:
=> select * from repository where prefix like 'epel-9.4'; id | name | prefix | category_id | version_id | arch_id | directory_id | disabled ----------+----------------------------------------------------+----------+-------------+------------+---------+--------------+---------- 27710620 | pub/archive/epel/9.4-2024-11-12/Everything/x86_64 | epel-9.4 | 4 | 934261 | 3 | 834505 | f 27710621 | pub/archive/epel/9.4-2024-11-12/Everything/s390x | epel-9.4 | 4 | 934261 | 9 | 833413 | f 27710622 | pub/archive/epel/9.4-2024-11-12/Everything/aarch64 | epel-9.4 | 4 | 934261 | 14 | 834481 | f 27710623 | pub/archive/epel/9.4-2024-11-12/Everything/ppc64le | epel-9.4 | 4 | 934261 | 15 | 833011 | f (4 rows)
Thanks! I'll move them to the Fedora Archive category then.
Not necessary. They have been created in the archive category. This should be solved now for 9.4. We need to do it again once 9.5 is archived.
Not sure I understand, in your SQL paste category_id 4 is EPEL, Fedora archive is category_id 12.
category_id
Ah, maybe I selected the wrong id, thanks. Let me check.
You already changed it. Good. Thank you. I was just following my old example which seems to have used the wrong ID. I did not check it.
Log in to comment on this ticket.