pungi.global.log
[LIVE_MEDIA ] [ERROR ] [FAIL] Live media (variant Labs, arch *, subvariant Scientific_KDE) failed, but going on anyway. [LIVE_MEDIA ] [ERROR ] Live media task failed: 43362314. See /mnt/koji/compose/32/Fedora-32-20200413.0/logs/x86_64/livemedia-Labs-Scientific_KDE.x86_64.log for more details.
Variant: Labs, subvariant: Scientific_KDE task failed. Pinging maintainers: @amitksaha
[LIVE_MEDIA ] [ERROR ] [FAIL] Live media (variant Labs, arch *, subvariant Robotics) failed, but going on anyway. [LIVE_MEDIA ] [ERROR ] Live media task failed: 43362335. See /mnt/koji/compose/32/Fedora-32-20200413.0/logs/x86_64/livemedia-Labs-Robotics.x86_64.log for more details.
[ERROR ] [FAIL] Image build (variant Container, arch armhfp, subvariant Container_Base) failed, but going on anyway. [IMAGE_BUILD ] [INFO ] [DONE ] Creating image (formats: docker, arches: aarch64-armhfp-ppc64le-s390x-x86_64, variant: Container, subvariant: Container_Base) (task id: 43362289)
[ERROR ] [FAIL] Image build (variant Container, arch armhfp, subvariant Container_Minimal_Base) failed, but going on anyway. [IMAGE_BUILD ] [INFO ] [DONE ] Creating image (formats: docker, arches: aarch64-armhfp-ppc64le-s390x-x86_64, variant: Container, subvariant: Container_Minimal_Base) (task id: 43362294)
[IMAGE_BUILD ] [ERROR ] [FAIL] Image build (variant Labs, arch *, subvariant Scientific) failed, but going on anyway. [IMAGE_BUILD ] [ERROR ] ImageBuild task failed: 43362317. See /mnt/koji/compose/32/Fedora-32-20200413.0/logs/x86_64/imagebuild-Labs-Scientific-vagrant-libvirt-vagrant-virtualbox.x86_64.log for more details.
Variant: Labs, subvariant: Scientific task failed. Pinging maintainers: @amitksaha
[ERROR ] [FAIL] Image build (variant Server, arch armhfp, subvariant Server) failed, but going on anyway. [IMAGE_BUILD ] [INFO ] [DONE ] Creating image (formats: raw-xz, arches: aarch64-armhfp, variant: Server, subvariant: Server) (task id: 43362320)
[ERROR ] [FAIL] Image build (variant Spins, arch armhfp, subvariant Minimal) failed, but going on anyway. [IMAGE_BUILD ] [INFO ] [DONE ] Creating image (formats: raw-xz, arches: aarch64-armhfp, variant: Spins, subvariant: Minimal) (task id: 43362328)
[ERROR ] [FAIL] Image build (variant Workstation, arch armhfp, subvariant Workstation) failed, but going on anyway. [IMAGE_BUILD ] [INFO ] [DONE ] Creating image (formats: raw-xz, arches: aarch64-armhfp, variant: Workstation, subvariant: Workstation) (task id: 43362343)
[ERROR ] Compose run failed: Live media task failed: 43362370. See /mnt/koji/compose/32/Fedora-32-20200413.0/logs/ppc64le-x86_64/livemedia-Workstation-Workstation.ppc64le-x86_64.log for more details.
Compose Total time: 5:40:47 Compose phase INIT time: 0:00:36. Compose phase PKGSET time: 0:57:23. Compose phase WEAVER: FAILED. Compose phase BUILDINSTALL time: 1:20:40. Compose phase GATHER time: 1:23:43. Compose phase OSTREE time: 0:44:21. Compose phase OSTREE_INSTALLER time: 1:05:56. Compose phase CREATEREPO time: 1:29:05. Compose phase CREATEISO time: 0:29:51. Compose phase IMAGE_BUILD time: 1:46:52. Compose phase LIVE_IMAGES time: 1:11:50. Compose phase LIVE_MEDIA: FAILED.
@lsedlar do you see anything in our pungi config that would cause this?
should this: image_name_format = '%(release_short)s-%(variant)s-%(disc_type)s-%(arch)s-%(version)s-1.1.iso' be: image_name_format = '%(release_short)s-%(variant)s-%(disc_type)s-%(arch)s-%(version)s-%(label).iso'
Is the question about https://koji.fedoraproject.org/koji/taskinfo?taskID=43362370 ?
Changing the name pattern would not have any effect on it. There's an NVR conflict in Koji. However the build https://koji.fedoraproject.org/koji/buildinfo?buildID=1493047 is created by the failed task https://koji.fedoraproject.org/koji/taskinfo?taskID=43362370 It seems to me like the task created the build and failed because the build exists already. @tkopecek, can you take a look?
Hmm, interesting. I agree, that problem is on koji's side. @kevin can you find something more about that error in the hub + builder log?
From our discussion in #koji channel, I am restarting the compose:
[04:41:23] <mboddu> tkopecek: Can you take a look at https://pagure.io/releng/failed-composes/issue/1294#comment-641852? [04:41:47] <tkopecek> mboddu: yes, added comment there few minutes ago [04:42:10] <mboddu> tkopecek: Oh, thanks :) [04:42:12] »» mboddu checks [04:51:57] <mboddu> tkopecek: https://paste.centos.org/view/d5b9175a is what I see on the builder [04:52:02] »» mboddu looking in the hub now [04:55:24] <tkopecek> mboddu: Could it be, that that builder was restarted meanwhile? It looks like task 43362370 was started twice. [04:56:17] <tkopecek> mboddu: Also times looks, like this happened. Task was (re)started about an hour after it was created. [04:57:13] <mboddu> tkopecek: The builder isn't restarted in last 22 days [04:57:14] <mboddu> $ uptime [04:57:14] <mboddu> 08:56:32 up 22 days, 10:17, 1 user, load average: 0.56, 1.44, 1.65 [04:59:03] <tkopecek> mboddu: Hmm, it could have been different one from the same channel. It could be found by grepping all kojid logs there for 43362370. Question is if it is worth it. [05:03:59] <mboddu> tkopecek: "there" as in the builder? [05:04:20] <mboddu> tkopecek: "Task was (re)started about an hour after it was created." where are you seeing this? [05:04:22] <tkopecek> mboddu: all builders in livemedia channel [05:04:38] <mboddu> Yeah, now that makes sense [05:05:05] <tkopecek> mboddu: https://koji.fedoraproject.org/koji/taskinfo?taskID=43362370 (created vs. started time) [05:06:06] <mboddu> tkopecek: Nah, that is fine, pungi spins off all the tasks and since we dont have builders to take care of all the tasks at one time, they will wait until they were able to get a builder [05:06:59] <mboddu> So, all the livemedia channel builders were busy until then [05:08:42] <mboddu> tkopecek: Also, it seems it happened with that particular task as other iso build from the same compose seems to be fine - https://koji.fedoraproject.org/koji/buildinfo?buildID=1493040 [05:09:08] <mboddu> tkopecek: What do you suggest we should do now? [05:10:19] <tkopecek> mboddu: Ok. Anyway, log says, that it failed in initImageBuild, which is run before individual arches (which are already closed). For fixing the compose, I would just restart it (I believe it will pass), for finding root cause I would run that grep over livemedia channel. [05:12:53] <mboddu> tkopecek: restart it as in restart the task or restart the compose? [05:13:14] <tkopecek> mboddu: compose
File koji issue: https://pagure.io/koji/issue/2145
Metadata Update from @humaton: - Issue status updated to: Closed (was: Open)
Log in to comment on this ticket.