dusty / failed-composes

Clone
Source Code
GIT

#152 Fedora-Rawhide-20180411.n.0 DOOMED
Opened 6 years ago by dustymabe. Modified 6 years ago

Open
Close issue as:
fixed invalid timeout killed known_issue

pungi.global.log

[INFO    ] [FAIL] Buildinstall (variant Everything, arch ppc64) failed, but going on anyway.
[INFO    ] Runroot task failed: 26301193. See /mnt/koji/compose/rawhide/Fedora-Rawhide-20180411.n.0/logs/ppc64/buildinstall-Everything.ppc64.log for more details.

[INFO    ] [FAIL] Buildinstall (variant Server, arch ppc64) failed, but going on anyway.
[INFO    ] Runroot task failed: 26301197. See /mnt/koji/compose/rawhide/Fedora-Rawhide-20180411.n.0/logs/ppc64/buildinstall-Server.ppc64.log for more details.
  • Compose run failed because: - No Task ID, look at log statement
[ERROR   ] Compose run failed: ERROR running command: pungi-gather --config=/mnt/koji/compose/rawhide/Fedora-Rawhide-20180411.n.0/work/ppc64/pungi/Server.ppc64.comps.conf --greedy=build --arch=ppc64p7 --profiler

The last error is strange. There should be log here, but the file is empty. It might be worth checking system logs on the machine if there was some other possible issue.

The error is in ppc64 buildinstall. In buildinstall-Everything.ppc64.log (and also the Server log) we see:

018-04-11 08:05:33,184: running ppc.tmpl
running ppc.tmpl
2018-04-11 08:05:41,189: command output:
/var/tmp/lorax.jjyru963/installroot/usr/sbin/wrapper: line 358: /scripts/file-size.sh: No such file or directory

command output:
/var/tmp/lorax.jjyru963/installroot/usr/sbin/wrapper: line 358: /scripts/file-size.sh: No such file or directory

2018-04-11 08:05:41,190: command returned failure (127)
command returned failure (127)
2018-04-11 08:05:41,190: template command error in ppc.tmpl:
template command error in ppc.tmpl:
2018-04-11 08:05:41,190:   runcmd /var/tmp/lorax.jjyru963/installroot/usr/sbin/wrapper -p of -D /var/tmp/lorax.jjyru963/installroot/usr/lib64/kernel-wrapper -i /mnt/koji/compose/rawhide/Fedora-Rawhide-20180411.n.0/work/ppc64/buildinstall/Everything/ppc/ppc64/initrd.img /mnt/koji/compose/rawhide/Fedora-Rawhide-20180411.n.0/work/ppc64/buildinstall/Everything/ppc/ppc64/vmlinuz -o /mnt/koji/compose/rawhide/Fedora-Rawhide-20180411.n.0/work/ppc64/buildinstall/Everything/images/netboot/ppc64.img
  runcmd /var/tmp/lorax.jjyru963/installroot/usr/sbin/wrapper -p of -D /var/tmp/lorax.jjyru963/installroot/usr/lib64/kernel-wrapper -i /mnt/koji/compose/rawhide/Fedora-Rawhide-20180411.n.0/work/ppc64/buildinstall/Everything/ppc/ppc64/initrd.img /mnt/koji/compose/rawhide/Fedora-Rawhide-20180411.n.0/work/ppc64/buildinstall/Everything/ppc/ppc64/vmlinuz -o /mnt/koji/compose/rawhide/Fedora-Rawhide-20180411.n.0/work/ppc64/buildinstall/Everything/images/netboot/ppc64.img
2018-04-11 08:05:41,215:   subprocess.CalledProcessError: Command '['/var/tmp/lorax.jjyru963/installroot/usr/sbin/wrapper', '-p', 'of', '-D', '/var/tmp/lorax.jjyru963/installroot/usr/lib64/kernel-wrapper', '-i', '/mnt/koji/compose/rawhide/Fedora-Rawhide-20180411.n.0/work/ppc64/buildinstall/Everything/ppc/ppc64/initrd.img', '/mnt/koji/compose/rawhide/Fedora-Rawhide-20180411.n.0/work/ppc64/buildinstall/Everything/ppc/ppc64/vmlinuz', '-o', '/mnt/koji/compose/rawhide/Fedora-Rawhide-20180411.n.0/work/ppc64/buildinstall/Everything/images/netboot/ppc64.img']' returned non-zero exit status 127.
  subprocess.CalledProcessError: Command '['/var/tmp/lorax.jjyru963/installroot/usr/sbin/wrapper', '-p', 'of', '-D', '/var/tmp/lorax.jjyru963/installroot/usr/lib64/kernel-wrapper', '-i', '/mnt/koji/compose/rawhide/Fedora-Rawhide-20180411.n.0/work/ppc64/buildinstall/Everything/ppc/ppc64/initrd.img', '/mnt/koji/compose/rawhide/Fedora-Rawhide-20180411.n.0/work/ppc64/buildinstall/Everything/ppc/ppc64/vmlinuz', '-o', '/mnt/koji/compose/rawhide/Fedora-Rawhide-20180411.n.0/work/ppc64/buildinstall/Everything/images/netboot/ppc64.img']' returned non-zero exit status 127.
Traceback (most recent call last):
  File "/usr/sbin/lorax", line 279, in <module>
    main()
  File "/usr/sbin/lorax", line 135, in main
    remove_temp=True, verify=opts.verify)
  File "/usr/lib/python3.6/site-packages/pylorax/__init__.py", line 359, in run
    treebuilder.build()
  File "/usr/lib/python3.6/site-packages/pylorax/treebuilder.py", line 312, in build
    self._runner.run(templatefile, kernels=self.kernels)
  File "/usr/lib/python3.6/site-packages/pylorax/ltmpl.py", line 220, in run
    self._run(commands)
  File "/usr/lib/python3.6/site-packages/pylorax/ltmpl.py", line 239, in _run
    f(*args)
  File "/usr/lib/python3.6/site-packages/pylorax/ltmpl.py", line 515, in runcmd
    stdout = runcmd_output(cmd)
  File "/usr/lib/python3.6/site-packages/pylorax/executils.py", line 347, in runcmd_output
    return execWithCapture(cmd[0], cmd[1:], **kwargs)
  File "/usr/lib/python3.6/site-packages/pylorax/executils.py", line 249, in execWithCapture
    reset_handlers=reset_handlers, reset_lang=reset_lang)[1]
  File "/usr/lib/python3.6/site-packages/pylorax/executils.py", line 201, in _run_program
    raise subprocess.CalledProcessError(proc.returncode, argv, output)
subprocess.CalledProcessError: Command '['/var/tmp/lorax.jjyru963/installroot/usr/sbin/wrapper', '-p', 'of', '-D', '/var/tmp/lorax.jjyru963/installroot/usr/lib64/kernel-wrapper', '-i', '/mnt/koji/compose/rawhide/Fedora-Rawhide-20180411.n.0/work/ppc64/buildinstall/Everything/ppc/ppc64/initrd.img', '/mnt/koji/compose/rawhide/Fedora-Rawhide-20180411.n.0/work/ppc64/buildinstall/Everything/ppc/ppc64/vmlinuz', '-o', '/mnt/koji/compose/rawhide/Fedora-Rawhide-20180411.n.0/work/ppc64/buildinstall/Everything/images/netboot/ppc64.img']' returned non-zero exit status 127.

The same basic error is also shown in lorax.log, program.log and pylorax.log in the buildinstall-Server-logs and buildinstall-Everything-logs subdirectories.

ppc64le logs look fine, though.

Hmm, actually, that can't be the whole story. The same thing failed in the 20180409.n.0 compose , but the log file is populated there, and the compose succeeded. So there's gotta be a bit more to this...

Filed the wrapper bug anyway, as it's clearly a bug and breaking ppc64 parts of the compose even if it's not actually making the overall compose fail.

The empty log is explained by a segfault. That could be some fluke, or maybe a bug in pungi or possibly in dnf. We don't have much information to debug this.

[Wed Apr 11 08:10:21 2018] pungi-gather[31042]: segfault at 55c0400a0073 ip 000055c0400a0073 sp 00007ffed2cd5f28 error 15

Login to comment on this ticket.

Metadata
None
None
None
None
Powered by Pagure 5.13.3
Documentation • File an Issue • About • SSH Hostkey/Fingerprint
© Red Hat, Inc. and others.