https://mirrors.fedoraproject.org/mirrorlist?repo=updates-released-modular-f32&arch=ppc64le gives URLs that returns 404s. It points to the path fedora-secondary/updates/32/Modular/ppc64le/Packages/os/, but that path does not exist; there is no os/ subdirectory of the Packages directory.
fedora-secondary/updates/32/Modular/ppc64le/Packages/os/
os/
Packages
By comparison with the results for other secondary arches and the results for Fedora 31, the Packages/os/ component should be dropped, the path should be fedora-secondary/updates/32/Modular/ppc64le/.
Packages/os/
fedora-secondary/updates/32/Modular/ppc64le/
This is urgent, it breaks openQA ppc64le update testing.
@adrian is this mm misdetecting? Or releng / pungi putting things in the wrong place? Or both?
Metadata Update from @kevin: - Issue priority set to: Waiting on Assignee (was: Needs Review) - Issue tagged with: mirrorlists
@adrian is this mm misdetecting?
Unfortunately this is the same error as every year.
Or releng / pungi putting things in the wrong place? Or both?
Not sure, for me it seems, that some tool creates an empty repository before any actual RPMs are available. This repository is detected by MirrorManager. Later the actual RPMs are put in a slightly different place. Once MirrorManager has detected a repository, updates-released-modular-f32 in this case, it will not change it by itself. So we somehow need to make sure that the empty repositories (repodata/repomd.xml) are put at the right location or this will happen every release. We are fixing things at least since F29 I think. The same happens every 6 months.
I have removed all ppc64le repositories from MM and reset all ctimes in the database for ppc64le and MM should now point to the correct repositories. I am sure this we will get more reports like this in the next weeks.
Let me know here if it does not work in 2 hours.
Metadata Update from @adrian: - Issue close_status updated to: Fixed - Issue status updated to: Closed (was: Open)
We make/made these empty repos. It might be that the 'normal' updates one and the modular are different?
https://kojipkgs.fedoraproject.org/compose/updates/empty-repo/
https://kojipkgs.fedoraproject.org/compose/updates/empty-epel-repo/
We should figure out the difference here and fix the empty repos for good.
For both trees we should do mv source/tree SRPMS. That is definitely wrong.
mv source/tree SRPMS
I think we should also remove ppc64le/os/Packages/debug and ppc64le/os/Packages/os.
ppc64le/os/Packages/debug
ppc64le/os/Packages/os
It also seems that all debug directory trees are wrong. The empty repositories always have <arch>/debug/tree but we actually always have debug in <arch>/debug/Packages and <arch>/debug/repodata. On the master mirror this is all gone and correct, but there is still the chance that MM detected some of them incorrectly, when they initially were being pushed out.
<arch>/debug/tree
<arch>/debug/Packages
<arch>/debug/repodata
Lot of things to clean up, not sure I caught all of them. I am also confused about the os in the path in the empty repositories. os does not appear in the updates tree.
os
ppc64le also has subdirectories in drpms which x86_64 does not have. s390x does not have any drpms directories. We do not see many problems with x86_64 empty tree maybe use that as an example for all the others.
Lets continue in https://pagure.io/releng/issue/9429
and try and fix this before f33 branching.
Login to comment on this ticket.