releng / failed-composes

#1530 Fedora-Rawhide-20200625.n.0 DOOMED
Closed by humaton. Opened by releng.

Closed
Reopen Issue

pungi.global.log

[OSTREE_INSTALLER] [ERROR   ] [FAIL] Ostree installer (variant Silverblue, arch x86_64) failed, but going on anyway.
[OSTREE_INSTALLER] [ERROR   ] Runroot task failed: 46126343. See /mnt/koji/compose/rawhide/Fedora-Rawhide-20200625.n.0/logs/x86_64/Silverblue/ostree_installer-3/runroot.log for more details.

[LIVE_MEDIA      ] [ERROR   ] [FAIL] Live media (variant Labs, arch *, subvariant Astronomy_KDE) failed, but going on anyway.
[LIVE_MEDIA      ] [ERROR   ] Live media task failed: 46131252. See /mnt/koji/compose/rawhide/Fedora-Rawhide-20200625.n.0/logs/x86_64/livemedia-Labs-Astronomy_KDE.x86_64.log for more details.

Variant: Labs, subvariant: Astronomy_KDE task failed. Pinging maintainers: @lupinix 

[LIVE_IMAGES     ] [ERROR   ] [FAIL] Live (variant Labs, arch armhfp, subvariant Python_Classroom) failed, but going on anyway.
[LIVE_IMAGES     ] [ERROR   ] LiveImage task failed: 46131249. See /mnt/koji/compose/rawhide/Fedora-Rawhide-20200625.n.0/logs/armhfp/liveimage-Fedora-Python-Classroom-armhfp-Rawhide-20200625.n.0.armhfp.log for more details.

[LIVE_MEDIA      ] [ERROR   ] [FAIL] Live media (variant Labs, arch *, subvariant Design_suite) failed, but going on anyway.
[LIVE_MEDIA      ] [ERROR   ] Live media task failed: 46131260. See /mnt/koji/compose/rawhide/Fedora-Rawhide-20200625.n.0/logs/x86_64/livemedia-Labs-Design_suite.x86_64.log for more details.

Variant: Labs, subvariant: Design_suite task failed. Pinging maintainers: @luya 

[LIVE_MEDIA      ] [ERROR   ] [FAIL] Live media (variant Labs, arch *, subvariant Scientific_KDE) failed, but going on anyway.
[LIVE_MEDIA      ] [ERROR   ] Live media task failed: 46131263. See /mnt/koji/compose/rawhide/Fedora-Rawhide-20200625.n.0/logs/x86_64/livemedia-Labs-Scientific_KDE.x86_64.log for more details.

Variant: Labs, subvariant: Scientific_KDE task failed. Pinging maintainers: @scitech 

[LIVE_MEDIA      ] [ERROR   ] [FAIL] Live media (variant Labs, arch *, subvariant Robotics) failed, but going on anyway.
[LIVE_MEDIA      ] [ERROR   ] Live media task failed: 46131291. See /mnt/koji/compose/rawhide/Fedora-Rawhide-20200625.n.0/logs/x86_64/livemedia-Labs-Robotics.x86_64.log for more details.

[LIVE_MEDIA      ] [ERROR   ] [FAIL] Live media (variant Labs, arch *, subvariant Games) failed, but going on anyway.
[LIVE_MEDIA      ] [ERROR   ] Live media task failed: 46131274. See /mnt/koji/compose/rawhide/Fedora-Rawhide-20200625.n.0/logs/x86_64/livemedia-Labs-Games.x86_64.log for more details.

Variant: Labs, subvariant: Games task failed. Pinging maintainers: @dagostinelli 

[LIVE_MEDIA      ] [ERROR   ] [FAIL] Live media (variant Labs, arch *, subvariant Python_Classroom) failed, but going on anyway.
[LIVE_MEDIA      ] [ERROR   ] Live media task failed: 46131293. See /mnt/koji/compose/rawhide/Fedora-Rawhide-20200625.n.0/logs/x86_64/livemedia-Labs-Python_Classroom.x86_64.log for more details.

Variant: Labs, subvariant: Python_Classroom task failed. Pinging maintainers: @churchyard @python-sig 

[ERROR   ] [FAIL] Image build (variant Cloud, arch ppc64le, subvariant Cloud_Base) failed, but going on anyway.
[IMAGE_BUILD     ] [INFO    ] [DONE ] Creating image (formats: qcow2-raw-xz-vmdk, arches: aarch64-ppc64le-s390x-x86_64, variant: Cloud, subvariant: Cloud_Base) (task id: 46131258)

[ERROR   ] [FAIL] Image build (variant Cloud, arch s390x, subvariant Cloud_Base) failed, but going on anyway.
[IMAGE_BUILD     ] [INFO    ] [DONE ] Creating image (formats: qcow2-raw-xz-vmdk, arches: aarch64-ppc64le-s390x-x86_64, variant: Cloud, subvariant: Cloud_Base) (task id: 46131258)

[IMAGE_BUILD     ] [ERROR   ] [FAIL] Image build (variant Labs, arch *, subvariant Scientific) failed, but going on anyway.
[IMAGE_BUILD     ] [ERROR   ] ImageBuild task failed: 46131326. See /mnt/koji/compose/rawhide/Fedora-Rawhide-20200625.n.0/logs/x86_64/imagebuild-Labs-Scientific-vagrant-libvirt-vagrant-virtualbox.x86_64.log for more details.

Variant: Labs, subvariant: Scientific task failed. Pinging maintainers: @scitech 

[ERROR   ] [FAIL] Image build (variant Container, arch s390x, subvariant Container_Base) failed, but going on anyway.
[IMAGE_BUILD     ] [INFO    ] [DONE ] Creating image (formats: docker, arches: aarch64-armhfp-ppc64le-s390x-x86_64, variant: Container, subvariant: Container_Base) (task id: 46131296)

[ERROR   ] [FAIL] Image build (variant Container, arch armhfp, subvariant Container_Base) failed, but going on anyway.
[IMAGE_BUILD     ] [INFO    ] [DONE ] Creating image (formats: docker, arches: aarch64-armhfp-ppc64le-s390x-x86_64, variant: Container, subvariant: Container_Base) (task id: 46131296)

[ERROR   ] [FAIL] Image build (variant Container, arch armhfp, subvariant Container_Minimal_Base) failed, but going on anyway.
[IMAGE_BUILD     ] [INFO    ] [DONE ] Creating image (formats: docker, arches: aarch64-armhfp-ppc64le-s390x-x86_64, variant: Container, subvariant: Container_Minimal_Base) (task id: 46131306)

[ERROR   ] [FAIL] Image build (variant Container, arch s390x, subvariant Container_Minimal_Base) failed, but going on anyway.
[IMAGE_BUILD     ] [INFO    ] [DONE ] Creating image (formats: docker, arches: aarch64-armhfp-ppc64le-s390x-x86_64, variant: Container, subvariant: Container_Minimal_Base) (task id: 46131306)

[ERROR   ] [FAIL] Image build (variant Server, arch armhfp, subvariant Server) failed, but going on anyway.
[IMAGE_BUILD     ] [INFO    ] [DONE ] Creating image (formats: raw-xz, arches: aarch64-armhfp, variant: Server, subvariant: Server) (task id: 46131328)

[ERROR   ] [FAIL] Image build (variant Spins, arch armhfp, subvariant Minimal) failed, but going on anyway.
[IMAGE_BUILD     ] [INFO    ] [DONE ] Creating image (formats: raw-xz, arches: aarch64-armhfp, variant: Spins, subvariant: Minimal) (task id: 46131336)

[IMAGE_BUILD     ] [ERROR   ] [FAIL] Image build (variant Spins, arch *, subvariant Xfce) failed, but going on anyway.
[IMAGE_BUILD     ] [ERROR   ] ImageBuild task failed: 46131349. See /mnt/koji/compose/rawhide/Fedora-Rawhide-20200625.n.0/logs/aarch64/imagebuild-Spins-Xfce-raw-xz.aarch64.log for more details.

Variant: Spins, subvariant: Xfce task failed. Pinging maintainers: @nonamedotc @maxamillion @kevin 

[ERROR   ] [FAIL] Image build (variant Workstation, arch armhfp, subvariant Workstation) failed, but going on anyway.
[IMAGE_BUILD     ] [INFO    ] [DONE ] Creating image (formats: raw-xz, arches: aarch64-armhfp, variant: Workstation, subvariant: Workstation) (task id: 46131354)

[LIVE_MEDIA      ] [ERROR   ] [FAIL] Live media (variant Spins, arch *, subvariant Xfce) failed, but going on anyway.
[LIVE_MEDIA      ] [ERROR   ] Live media task failed: 46131298. See /mnt/koji/compose/rawhide/Fedora-Rawhide-20200625.n.0/logs/x86_64/livemedia-Spins-Xfce.x86_64.log for more details.

Variant: Spins, subvariant: Xfce task failed. Pinging maintainers: @nonamedotc @maxamillion @kevin 

[LIVE_MEDIA      ] [ERROR   ] [FAIL] Live media (variant Spins, arch *, subvariant LXQt) failed, but going on anyway.
[LIVE_MEDIA      ] [ERROR   ] Live media task failed: 46131316. See /mnt/koji/compose/rawhide/Fedora-Rawhide-20200625.n.0/logs/x86_64/livemedia-Spins-LXQt.x86_64.log for more details.

Variant: Spins, subvariant: LXQt task failed. Pinging maintainers: @lupinix @zsun 

[LIVE_MEDIA      ] [ERROR   ] [FAIL] Live media (variant Spins, arch *, subvariant Mate) failed, but going on anyway.
[LIVE_MEDIA      ] [ERROR   ] Live media task failed: 46131315. See /mnt/koji/compose/rawhide/Fedora-Rawhide-20200625.n.0/logs/x86_64/livemedia-Spins-Mate.x86_64.log for more details.

[LIVE_MEDIA      ] [ERROR   ] [FAIL] Live media (variant Spins, arch *, subvariant LXDE) failed, but going on anyway.
[LIVE_MEDIA      ] [ERROR   ] Live media task failed: 46131308. See /mnt/koji/compose/rawhide/Fedora-Rawhide-20200625.n.0/logs/x86_64/livemedia-Spins-LXDE.x86_64.log for more details.

Variant: Spins, subvariant: LXDE task failed. Pinging maintainers: @cwickert 

[LIVE_MEDIA      ] [ERROR   ] [FAIL] Live media (variant Spins, arch *, subvariant Cinnamon) failed, but going on anyway.
[LIVE_MEDIA      ] [ERROR   ] Live media task failed: 46131307. See /mnt/koji/compose/rawhide/Fedora-Rawhide-20200625.n.0/logs/x86_64/livemedia-Spins-Cinnamon.x86_64.log for more details.

Variant: Spins, subvariant: Cinnamon task failed. Pinging maintainers: @grinnz 

[LIVE_MEDIA      ] [ERROR   ] [FAIL] Live media (variant Labs, arch *, subvariant Jam_KDE) failed, but going on anyway.
[LIVE_MEDIA      ] [ERROR   ] Live media task failed: 46131287. See /mnt/koji/compose/rawhide/Fedora-Rawhide-20200625.n.0/logs/x86_64/livemedia-Labs-Jam_KDE.x86_64.log for more details.

Variant: Labs, subvariant: Jam_KDE task failed. Pinging maintainers: @eeickmeyer 

[LIVE_MEDIA      ] [ERROR   ] [FAIL] Live media (variant Labs, arch *, subvariant Security) failed, but going on anyway.
[LIVE_MEDIA      ] [ERROR   ] Live media task failed: 46131278. See /mnt/koji/compose/rawhide/Fedora-Rawhide-20200625.n.0/logs/x86_64/livemedia-Labs-Security.x86_64.log for more details.

Variant: Labs, subvariant: Security task failed. Pinging maintainers: @jsimon @fab

  • Compose run failed because: - 46131297
[ERROR   ] Compose run failed: Live media task failed: 46131297. See /mnt/koji/compose/rawhide/Fedora-Rawhide-20200625.n.0/logs/x86_64/livemedia-Spins-KDE.x86_64.log for more details.

Compose Total time: 17:12:58
Compose phase INIT time: 0:00:25.
Compose phase PKGSET time: 0:28:03.
Compose phase WEAVER: FAILED.
Compose phase BUILDINSTALL time: 1:57:37.
Compose phase GATHER time: 2:56:13.
Compose phase OSTREE time: 0:29:49.
Compose phase OSTREE_INSTALLER time: 0:59:51.
Compose phase CREATEREPO time: 0:24:17.
Compose phase REPOCLOSURE time: 0:02:21.
Compose phase CREATEISO time: 0:13:53.
Compose phase LIVE_IMAGES time: 0:38:04.
Compose phase IMAGE_BUILD time: 4:58:25.
Compose phase LIVE_MEDIA: FAILED.

So, almost all live images failed because of the selinux-policy-targeted %post script failing. That is happening because it fails if /etc/selinux/config does not exist when it is run.

In today's compose, for some reason - most likely this new version of RPM - in most live image builds, selinux-policy-targeted got ordered for install before selinux-policy. Previously, it seems to have always been ordered after selinux-policy. /etc/selinux/config is created by selinux-policy's %post script.

I have sent a pull request that fixes various scriptlets in selinux-policy to work even if /etc/selinux/config is not present, but we could not merge and build selinux-policy because a commit has mysteriously gone missing from its dist-git repo. The most recent build of selinux-policy is selinux-policy-3.14.6-15.fc33 from 2020-06-05, but the most recent commit in its repo is the one for -14 from 2020-05-19. We have no idea how the -15 commit went missing, but we don't want to bodge up a push over it at least not yet, we're hoping @zpytela still has the commit locally and can re-push it.

As to the ordering issue: well, there is a loop here (the more we look, the more dependency loops we find):

  1. selinux-policy Requires: rpm-plugin-selinux
  2. rpm-plugin-selinux Requires: selinux-policy-base, which is a virtual provide that selinux-policy-targeted, selinux-policy-mls and selinux-policy-minimum all provide; in this case, selinux-policy-targeted is providing it
  3. selinux-policy-targeted Requires(pre): selinux-policy

so RPM is obliged to break that loop somehow. (There may be other complicating dependencies too, that's just the core loop I found right away). I think in this case we expect the Requires(pre): dep to take precedence and so RPM should try hard to order selinux-policy ahead of selinux-policy-targeted, so this may be an RPM bug. Tagging @pmatilai for thoughts on that.
Edited by adamwill

@kevin found the missing commit in a pull request, so I was able to re-merge it from that branch and push it, then push my scriptlet fix on top of it. Build is running now.

Rpm ordering hasn't changed at all in recent times, this is just some new dependency added someplace that broke the balance of terror under which it usually gets ordered right. An install log with --deploops / RPMTRANS_FLAG_DEPLOOPS is usually the easiest way to figure out what's going on, especially if you can diff to a working version.

I'm off to vacation after today, talk to @ffesti for follow-through if needed.

For the record, here's a few follow-up bug reports:

  • https://bugzilla.redhat.com/show_bug.cgi?id=1851558
  • https://bugzilla.redhat.com/show_bug.cgi?id=1851268
  • https://bugzilla.redhat.com/show_bug.cgi?id=1851266

Metadata Update from @humaton:
- Issue status updated to: Closed (was: Open)
Metadata
None

DOOMED Fedora Rawhide

None
None
None
Static archive. pagure.io was sunset in 2026; this is a read-only snapshot.
Red Hat Logo

Fedora is sponsored by Red Hat. Learn more about the relationship between Red Hat and Fedora »

© Red Hat, Inc. and others.