Our currently sign+push process was designed to automate signing rpm packages, and generating repositories (signed repodata too). As some SIGs would like to start build images (.iso images, live media, etc), we should start designing a function to plumb in that process that wouldn't even try to sign rpm packages, but rather do something else, like downloading produced media image, and push it correct location. That can also include other things like checksum files and detached signature on it
Metadata Update from @arrfab: - Issue marked as blocking: #958 - Issue marked as blocking: #971 - Issue marked as blocking: #988
Metadata Update from @arrfab: - Issue assigned to arrfab
Metadata Update from @arrfab: - Issue tagged with: cbs, centos-build-pipeline, centos-common-infra, feature-request, high-gain, high-trouble, mini-initiative
@tkopecek : simple question about kiwi For image task, koji considers that it's like a pkg, so easy to then tag-build to various tasks Example (on other koji) : https://koji.mbox.centos.org/koji/buildinfo?buildID=22804
image
Would that work the same for a kiwi build ? Ideally SIGs would just then have to tag-build to promote to other tags (which is triggering the sign+push process)
yes, it is exactly the same
Perfect ! so that means we can use the same thing but just look at some tags to see if they have rpms (and so signing these/importing/call distrepo task) or just media images and so just download build and put it in place
@ngompa , @dcavalca , @tdawson : can you try a real (non --scratch one) build but at this stage let it be in -candidate tag (it's ignored at this stage by signing+push process and we can validate that it works ? I can then write/test a simple function to start moving things around in case of a media/image build for SIGs
-candidate
Sure, I'll give it a shot momentarily.
Non-scratch build fails due to unregistered package name? https://cbs.centos.org/koji/taskinfo?taskID=3265887
which sounds logic if that's considered a build ? so https://sigs.centos.org/guide/cbs/#submit-a-build-on-cbs would apply (and like for a rpm pkg, one would just have to add it to the tags)
Here's a new run after registering the "package": https://cbs.centos.org/koji/taskinfo?taskID=3278044
ngompa@fedora ~/S/p/c/kiwi-descriptions (c9s)> cbs add-pkg --owner=ngompa hyperscale9s-spin_media-experimental-candidate CentOS-Stream-Hyperscale-Spin-OpenStack Adding 1 packages to tag hyperscale9s-spin_media-experimental-candidate ngompa@fedora ~/S/p/c/kiwi-descriptions (c9s)> cbs add-pkg --owner=ngompa hyperscale9s-spin_media-experimental-testing CentOS-Stream-Hyperscale-Spin-OpenStack Adding 1 packages to tag hyperscale9s-spin_media-experimental-testing ngompa@fedora ~/S/p/c/kiwi-descriptions (c9s)> cbs add-pkg --owner=ngompa hyperscale9s-spin_media-experimental-release CentOS-Stream-Hyperscale-Spin-OpenStack Adding 1 packages to tag hyperscale9s-spin_media-experimental-release ngompa@fedora ~/S/p/c/kiwi-descriptions (c9s)> bash ~/Scripts/centos-cbs-build-c9s-hsx-spin-kiwi.sh kiwi-descriptions CentOS-Stream-Hyperscale-Spin.kiwi oem OpenStack 0.n.20230314 Created task: 3278044 Task info: https://cbs.centos.org/koji/taskinfo?taskID=3278044 Watching tasks (this may be safely interrupted)... 3278044 kiwiBuild (noarch): free 3278044 kiwiBuild (noarch): free -> open (x86-5.cbs.centos.org) 3278046 createKiwiImage (aarch64): free 3278045 createKiwiImage (x86_64): free 3278046 createKiwiImage (aarch64): free -> open (aarch64-01.rdu2.centos.org) 3278045 createKiwiImage (x86_64): free -> open (x86-5.cbs.centos.org)
It looks like Koji doesn't like the *.sha256 checksum files:
*.sha256
BuildError: Unsupported file type: CentOS-Stream-Hyperscale-Spin-OpenStack.x86_64-9.0.0-0.n.20230314.qcow2.sha256
@tkopecek ^ ? any idea about that ?
Filed https://pagure.io/koji/issue/3736 for the koji issue.
Metadata Update from @arrfab: - Assignee reset
@tkopecek , @dcavalca : any news on the kiwi plugin issue for koji ? trying to revisit some open tickets and what's blocking and this one is still on the list
I'll look at it today (slipped through my queue)
Ah, I've forgotten to add some hints to docs. Kiwi produces a few more archivetypes than they enabled by default. It could be updated via running:
koji -p cbs call addArchiveType kiwi-checksum "Kiwi package list checksum" sha256 koji -p cbs call addArchiveType kiwi-changes "Kiwi changes file" "changes.xz changes" koji -p cbs call addArchiveType kiwi-packages "Kiwi packages listing" packages koji -p cbs call addArchiveTYpe kiwi-verified "Kiwi verified package list" verified
These for additional types should be sufficient for now. Anyway, if some kiwi target produce something new it could be added same way.
Created also PR https://pagure.io/koji/pull-request/3775
Metadata Update from @arrfab: - Issue tagged with: centos-7-8s-eol
Thanks a lot ... Added manually through koji calls ! @ngompa: can you eventually give it another try ? If that works, I'll be able to play in .stg. env with a function to retrieve tagged builds and then we can decide on next steps for artifacts release automation
@ngompa kicked another kiwi build and this time it worked : https://cbs.centos.org/koji/buildinfo?buildID=43706
So I'll verify how to easily parse info and how to move artifacts somewhere
Metadata Update from @zlopez: - Issue priority set to: High Priority (was: Needs Review)
@ngompa : I had time to work on a function to process images builds and It's working in .stg. env. For example, this is the kind of layout we can expect to land on mirrors :
└── images-latest └── x86_64 ├── CentOS-Stream-Hyperscale-Appliance-OpenStack.x86_64-9.0.0-1.qcow2 ├── CentOS-Stream-Hyperscale-Appliance-OpenStack.x86_64-9.0.0-1.qcow2.sha256 ├── CentOS-Stream-Hyperscale-Appliance-OpenStack.x86_64-9.0.0-202305161312.qcow2 ├── CentOS-Stream-Hyperscale-Appliance-OpenStack.x86_64-9.0.0-202305161312.qcow2.sha256 ├── SHA256SUM └── SHA256SUM.asc
Dont' look at the names, it's just some tests I did and that's the name in the kiwi xml files So it would create a directory with <arch> and then like for rpm repositories (except that would automatically adding/removing tagged/untagged builds)
If that works for you, I'd like to push that change on the new (migrated) signing process. Worth reminding that we agreed to use (see altimage discussion). So that would mean that we'd just need to recreate new tags for hyperscale (but we have automated the process, based on work done for altimages tags)
This is now pushed to prod (after some testing in .stg. env). Closing this ticket and working in new ones for the new tags to be created for images builds
images
Metadata Update from @arrfab: - Issue close_status updated to: Fixed - Issue status updated to: Closed (was: Open)
Log in to comment on this ticket.