Imagebuilds are failing on ppc64le from past few days in our nightly composes. By looking at latest nightly composes, ppc64le imagebuild is failing for f28-{cloud,container}, f29-{cloud,container,AH} .
I looked into failure details for ppc64le AH image from F29-updates-testing, seems to work fine when ran imagefactory locally on ppc64le.
By looking at past F29-updates(-testing) nightly composes, it seems that this issue is happening since 22nd December.
Did something changed in/around ppc64le imagebuilder side in between 20th and 22nd December? Don't see any useful information from the imagebuild logs.
There were a few changes to builders done at that time, see ansible git log for more information. Notably, ppc64le builders were moved from Fedora 28 to Fedora 29
Hmm, https://bugzilla.redhat.com/show_bug.cgi?id=1652845#c14 is one of the issue we have seen when creating vm in F29, ppc64le with kvm_pr in nested virt. Is this the case for our builder?
Yes, our builders are KVM guests themselves and have kvm_pr module loaded.
Thanks for confirming! VM install works fine with qemu-2.12.0 . Maybe we can try downgrading qemu to 2.12.0 on ppc64le builders and see if it fixes the imagebuild in next compose?
Metadata Update from @mizdebsk: - Issue priority set to: Waiting on Assignee (was: Needs Review)
ok. I downgraded buildvm-ppc64le-01/02 to qemu-2.12.0-4.fc29. (01/02 are the only ones in the image and compose channels).
Thanks Kevin! Imagebuild from 20190103 looks good on ppc64le. Hopefully we will have qemu fix soon in order to not use this workaround.
Metadata Update from @sinnykumari: - Issue close_status updated to: Fixed - Issue status updated to: Closed (was: Open)
thanks for looking into this @sinnykumari @kevin @mizdebsk
Login to comment on this ticket.