#1411 CBS: Repositories not synchronized
Closed: Fixed with Explanation a month ago by arrfab. Opened a month ago by pjgeorg.

Since a few days there seem to be some issues how CBS gets updated repositories.
E.g., see 1, which is failing for a few days now due to missing packages, although these missing packages have been added to mirror.stream.centos.org few days ago. Looking into details, it seems the koji internal repositories are not being updated anymore for some tags.

E.g., comparing kmods9s-packages-main-el9s-build to kmods9s-packages-rebuild-el9s-build the former shows 5.14.0-444.el9 as latest version for the kernel package, the latter 5.14.0-446.el9.

Note: The available kernel version was obtained by running
dnf --nogpgcheck --repofrompath cbs,https://cbs.centos.org/kojifiles/repos/kmods9s-packages-main-el9s-build/latest/x86_64/ --repo=cbs list kernel

Metadata Update from @arrfab:
- Issue assigned to arrfab

a month ago

Metadata Update from @arrfab:
- Issue tagged with: cbs, centos-common-infra, high-gain, medium-trouble

a month ago

I had a quick look and while the internal mirror itself is up2date (we have monitoring in place for that one too), it's more at the kojihub/kojira level that it seems it was confused, but don't know why.
example :

May 16 07:19:52  kojira[483643]: 2024-05-16 07:19:52,510 [WARNING] {483643} koji.repo.regen:734 Tag name/id mismatch: directory: storage8-common-el8-build, name: virt8-ovirt-common-el8-build, id: 1932

So somehow it was confused about which repo to watch for which tags, and that for multiple tags. I just relaunched kojira itself and it processed all repositories in normal order, including kicking some "regen-repo" tasks
Waiting for feedback but also keeping an eye on failing tasks if there is some underlying issues

hmm, some tasks are failing in a strange way : https://cbs.centos.org/koji/taskinfo?taskID=3960176

HTTP response code said error: The requested URL returned error: 503 Service Unavailable
Warning: Downloading of repodata failed: https://kojihub.stream.centos.org/kojifiles/repos/c9s-build/latest/x86_64/

But I do see that it was generated so wondering if kojira detects some changes but at same time we have upstream stream kojihub also replacing files

that should have been the issue as kicked again and it worked https://cbs.centos.org/koji/taskinfo?taskID=3960234 .. Let me reprocess all *el9s-build tags and verify that it would contain up2date pkgs

All *-el9s-build tags which include epel seem to be affected.
I remember reading about some issues due to changed compression used in Fedora infra, might this be related? I can not find the original post anymore, but here's one 1 related issue.

ouch : createrepo_c has an issue parsing epel9 repodata :

Critical: Cannot load repo: "https://dl.fedoraproject.org/pub/epel/next/9/Everything/x86_64/" : Error encountered while parsing:primary.xml parsing: Cannot open /tmp/koji/tasks/476/3960476/createrepo_c_tmp_repo_qLYHPA/repodata/4a517e26b96f3a03d135e5e58e93cafd21f912b5c72261520d6b53783f172859-primary.xml.zst: Cannot detect compression type

(from https://cbs.centos.org/kojifiles/work/tasks/476/3960476/mergerepos.log)

created ticket in Fedora infra to see what's possible

Upstream ticket closed (epel9-next now using correct compression method) and so all regen-repo tasks seem to have worked too : https://cbs.centos.org/koji/taskinfo?taskID=3961309

Metadata Update from @arrfab:
- Issue close_status updated to: Fixed with Explanation
- Issue status updated to: Closed (was: Open)

a month ago

Log in to comment on this ticket.