pungi.global.log
[ERROR ] [FAIL] Image build (variant AtomicHost, arch ppc64le, subvariant AtomicHost) failed, but going on anyway. [IMAGE_BUILD ] [INFO ] Hardlinking /mnt/koji/packages/Fedora-AtomicHost/29_Update/20190213.0/images/Fedora-AtomicHost-29_Update-20190213.0.aarch64.qcow2 to /mnt/koji/compose/updates/Fedora-29-updates-testing-20190213.0/compose/AtomicHost/aarch64/images/Fedora-AtomicHost-29_Update-20190213.0.aarch64.qcow2
Cause of ppc64le imagebuild failure is 2019-02-13T07:34:50.264169Z qemu-system-ppc64: cannot set up guest memory 'ppc_spapr.ram': Cannot allocate memory"
2019-02-13T07:34:50.264169Z qemu-system-ppc64: cannot set up guest memory 'ppc_spapr.ram': Cannot allocate memory"
It appears that memory was not sufficient on the ppc64le builder buildvm-ppc64le-01.ppc.fedoraproject.org at the time of this build . Respective koji build https://koji.fedoraproject.org/koji/taskinfo?taskID=32772956 . @kevin how is the current memory usage look like on builder buildvm-ppc64le-01.ppc.fedoraproject.org ? Do we need to free up some memory or next imagebuild should go fine?
buildvm-ppc64le-01.ppc.fedoraproject.org
[root@buildvm-ppc64le-01 ~][PROD]# free -m total used free shared buff/cache available Mem: 10204 9249 71 4 883 865 Swap: 2047 2047 0
The problem is that there are 3 old fedora 28 virt instances hanging around. It seems like they are failing, but not exiting cleanly.
I'll kill them and see if it helps any, but I think they might pile up again. ;(
It seems insufficient memory issues is happening quite frequently on both ppc64le image builders buildvm-ppc64le-02 (with Fedora-AtomicHost-29_Update-20190215.0) and buildvm-ppc64le-01 (with Fedora-AtomicHost-29-20190214.0)
I cleared all old qemu's from both:
buildvm-ppc64le-01.ppc.fedoraproject.org | CHANGED | rc=0 >> total used free shared buff/cache available Mem: 10204 1087 6644 3 2472 9027 Swap: 2047 88 1959 buildvm-ppc64le-02.ppc.fedoraproject.org | CHANGED | rc=0 >> total used free shared buff/cache available Mem: 10204 988 8526 2 690 9127 Swap: 2047 130 1917
I guess lets see what it looks like over the weekend...
Login to comment on this ticket.