Description:
Unable to makecache or update, EPEL only, bad meta from mirrors until meta is replaced wth createrepo.
Once in a blue moon the stars align and one mirrorserver will provide a repomd.xml with references to nonexistent "{hash}-[filelists,primary,prestodelta].xml.xz" files. Inevitably this clears up in a few hours at worst. Seven days ago (13 June 2025) this occurred, has persisted since then, and is uniform across the handful of NRLDC-local mirrors (Ashburn, Reston, Manassas, Weaton, Silver Spring, Washington DC). There is permission denied for various ".chromium-{version}.el8.src.rpm.{tmpstr}" (afaik already being worked) but no other errors for either rsync or wget pulls. I've spot-checked with a few of the servers' admins; they report same in their logs.
When needed: "Eventually." (low priority, obvious 'createrepo' mitigation works just fine)
When no longer needed: N/A
Impact if unaddressed: "I will be very upset. There may even be tears."
Can you show the full error you are getting ? Just repomd.xml not matching checksum?
From your mention of an el8 package, I guess this is epel8? but would be nice to know the exact repo and arch
Looks like it cleared itself up. Last pass with the issue was Sat 21 Jun 2025 06:48:06 PM EDT, the pass at Mon 23 Jun 2025 03:49:27 AM EDT did not have the issue. I guess this can be closed, unless you think the same bad meta showing up in multiple mirrors for just shy of 9 days should be looked into.
Details...
All permissions failures were for failed chromium artifacts. Looking back at previous passes over the last 36 hours shows the permission denied instances like:
rsync: send_files failed to open "/epel/8/Everything/source/tree/Packages/c/.chromium-128.0.6613.119-1.el8.src.rpm.EhYBtr" (in fedora): Permission denied (13) rsync: send_files failed to open "/epel/8/Everything/source/tree/Packages/c/.chromium-129.0.6668.58-1.el8.src.rpm.KjFdbH" (in fedora): Permission denied (13) rsync: send_files failed to open "/epel/8/Everything/source/tree/Packages/c/.chromium-129.0.6668.58-1.el8.src.rpm.talsHn" (in fedora): Permission denied (13) rsync: send_files failed to open "/epel/9/Everything/source/tree/Packages/c/.chromium-128.0.6613.137-1.el9.src.rpm.0PeNpR" (in fedora): Permission denied (13) rsync: send_files failed to open "/epel/9/Everything/source/tree/Packages/c/.chromium-129.0.6668.58-1.el9.src.rpm.BTCmN4" (in fedora): Permission denied (13) rsync: send_files failed to open "/epel/testing/8/Everything/source/tree/Packages/c/.chromium-129.0.6668.100-1.el8.src.rpm.DThQbh" (in fedora): Permission denied (13) rsync: send_files failed to open "/epel/testing/9/Everything/source/tree/Packages/c/.chromium-129.0.6668.100-1.el9.src.rpm.pMwE0X" (in fedora): Permission denied (13)
gradually reduced from 12-15 at time of the OP to 1 yesterday evening and none now. I don't have any way of knowing from this end whether this was related, but so far all failures have coincided with these errors in the prior repo rsync pass and the non-failure intervals before ~14 Jun and after ~0300 this morning do not have any such errors in their preceeding rsync pass.
All "bad meta" was in the epel/8/Everything/x86_64/repodata/repomd.xml file. The main issue that was blocking dnf from proceeding was location hrefs to nonexistent files in the filelists, primary, and prestodelta blocks. For example repomd.xml should be composed of blocks like:
<data type="{label}"> <checksum type="{method}">{hash}</checksum> <open-checksum type="{method}">{hash}</open-checksum> <location href="repodata/{hash}-{label}.xml.xz"/> <timestamp>{epoch}</timestamp> <size>{int}</size> <open-size>{int}</open-size> </data>
where "repodata/{hash}-{label}.xml.xz" exists relative to the parent dir of the repomd.xml file. For >8 days the repomd.xml (which was changing with each sync) contained blocks for {label} == "filelists", "primary", and "prestodata" whose location href did not match any file in the otherwise successfully sync'd repodata. This blocked dnf because the clients, installing into new system image/container filetrees with no prior cache, would pull the repomd.xml, then try to pull each of the contained blocks' location hrefs. That cache build would fail because for those three repomd clauses (filelists, primary, and prestodata) the associated href-indicated file didn't exist... at least, not until this morning around 0300 EST.
Metadata Update from @patrikp: - Issue close_status updated to: It's all good - Issue status updated to: Closed (was: Open)
Log in to comment on this ticket.