#233 Update mock and possibly the RPM stack on CBS builders
Closed: Fixed 2 years ago by arrfab. Opened 3 years ago by bstinson.

We've run into an issue where carets in the version of a BuildRequires/Requires causes trouble because the rpm stack is too old: https://cbs.centos.org/kojifiles/work/tasks/2865/2002865/root.log

There are 2 possible fixes:
- Update mock and start enabling the new nspawn chroot features (which use the rpm stack from the target distribution)
- Update the RPM stack directly on the builders to something that supports CaretsInVersion

SInce we'll need to get mock moving forward anyways, we should consider doing both. We're likely to follow the same pattern for koji.mbox.centos.org but that work is tracked in another place.

/cc: @dcavalca


You don't need to enable nspawn to use bootstrap chroots. That just requires a new enough mock... (we still use old chroot in fedora land)

@bstinson as you know, we're already using a higher rpm stack that the one from the distro on builders (running centos 7 but with higher rpm (https://cbs.centos.org/koji/buildinfo?buildID=26092 : rpm-4.13.0.2-1.el7.c8 vs rpm-4.11.3-45.el7 from distro)

Bumping that (so rebuilding newer rpm and distribute on builders) is something we can do (as that's what was done already for koji.mbox and cbs.centos.org builders) but probably tested somewhere first (to not introduce issues)
Which would be the rpm version we'd need to bump to (and that would work on el7 hosts that is) ?

Metadata Update from @arrfab:
- Issue priority set to: Waiting on Reporter (was: Needs Review)
- Issue tagged with: cbs, centos-stream, feature-request, medium-gain, medium-trouble, need-more-info

3 years ago

Hi, after quick internal discussion, I'll discuss with @alphacc to come with a different plan instead : as new kojihub for stream-9 will be running on top of Stream 8 with koji 1.23 , it would be a better idea to update cbs.centos.org koji setup to same version, and also update it from centos 7 to centos 8s .
We'll come with a plan and we'll announce on centos-devel list (and plan , based on priorities within the team, available time, etc)

Metadata Update from @arrfab:
- Issue assigned to arrfab

3 years ago

Metadata Update from @arrfab:
- Issue priority set to: Next Meetings (was: Waiting on Reporter)

3 years ago

Just verifying open tickets and trying to prioritize some work : is that ticket still needed ? as it seems there were some builds for systemd on cbs recently so just confirming that it's still something to be evaluated/worked on or not.
Thanks for feedback :)

We do still need this, because we want to be able to build snapshot versions of packages using the same features supported in RHEL/CentOS 8.

Ok so quick feedback on this one : my initial plan was just to update kojid instances one by one (from centos 7 to 8-stream) but after a quick test I realized that it's not possible.
What we have built through infra tags for buildtools is koji-1.23.0-1.el8.noarch (basically from epel8 branch) , while the version we have for el7/c7 is koji-1.19.1-3.el7.centos.noarch and they don't work together.
So if we want to update, we have to update everything to 8-stream, and so upgrade to common version 1.23.0 (there is no way to build that version for el7 due to python2 vs python3)

That means that it's not a simple action as we need to validate all other components, like centmsg plugin for mqtt/signing notifications, and other things too, and if that all works, we'll have to plan a whole upgrade (downtime for cbs). As it's more than just one hour here and one hour there, I'll have to see internally about prioritizing this request and if that's accepted as valid request, or just RFE

Just a quick update/feedback : as I also have to work on #285 , I discussed with @alphacc and deploying a parallel test setup (dev) to test the migration from c7/koji 1.19 to c8-stream/koji 1.23.
While we know that our ansible roles work fine, we just need to look at the specific tuned options/scripts for cbs, like centmsg plugin (using mqtt for signing bus) and verify what needs to be ported to python3.
Once we have all working in test, we'll just announce downtime and migration on centos-devel list

status update (pto tomorrow) : I deployed a kojihub/kojid setup with 8-stream and adapted some of our roles and koji glue scripts to work with python3 and a quick test showed us that it seems to be all working.
Next week I'll review with @alphacc and other tests and then we'll announce on centos-devel list the maintenance window for the whole cbs.centos.org upgrade from 7 to 8-stream

The whole cbs infra was (as announced) upgraded to koji 1.23 , and all nodes reinstalled on 8-stream, so closing this RFE

Metadata Update from @arrfab:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)

2 years ago

Login to comment on this ticket.

Metadata
Boards 1
CBS Status: Backlog