Issue #256: Fedora-27-updates-20180506.0 FINISHED_INCOMPLETE - failed-composes

dusty / failed-composes

#256 Fedora-27-updates-20180506.0 FINISHED_INCOMPLETE

Opened 5 years ago by dustymabe. Modified 5 years ago

pungi.global.log

26828036

[INFO    ] [FAIL] Ostree (variant Everything, arch ppc64le) failed, but going on anyway.
[INFO    ] Runroot task failed: 26828036. See /mnt/koji/compose/updates/Fedora-27-updates-20180506.0/logs/ppc64le/Everything/ostree-2/runroot.log for more details.

sinnykumari commented 5 years ago

I can create successfully ppc64le ostree on my local F27 box. Command I used was

$ sudo  pungi-make-ostree tree --repo=/home/skumari/ostree/atomic/repo/  --log-dir=/home/skumari/ostree/ppc64le_logs  --treefile=/home/skumari/ostree/fedora-atomic-host-updates-stable.json --extra-config=/home/skumari/ostree/extra_config.json

Downloaded required files locally from https://kojipkgs.fedoraproject.org/compose/updates/Fedora-27-updates-20180506.0/work/ostree-2/ , which is what used in this compose.

Coming back to task failure, by looking at root.log following is the error message:

DEBUG util.py:522:  Executing command: ['/usr/bin/systemd-nspawn', '-q', '-M', '5172e5707e4842ce8a760ff36fe96cd1', '-D', '/var/lib/mock/f27-build-12352048-906323/root', '-a', '--setenv=TERM=vt100', '--setenv=SHELL=/bin/bash', '--setenv=HOME=/builddir', '--setenv=HOSTNAME=mock', '--setenv=PATH=/usr/bin:/bin:/usr/sbin:/sbin', '--setenv=PROMPT_COMMAND=printf "\\033]0;<mock-chroot>\\007"', '--setenv=PS1=<mock-chroot> \\s-\\v\\$ ', '--setenv=LANG=en_US.UTF-8', '/bin/sh', '-c', '{ rm -f /var/lib/rpm/__db*; rm -rf /var/cache/yum/*; set -x; pungi-make-ostree tree --repo=/mnt/koji/compose/atomic/repo --log-dir=/mnt/koji/compose/updates/Fedora-27-updates-20180506.0/logs/ppc64le/Everything/ostree-2 --treefile=/mnt/koji/compose/updates/Fedora-27-updates-20180506.0/work/ostree-2/config_repo/fedora-atomic-host-updates-stable.json --extra-config=/mnt/koji/compose/updates/Fedora-27-updates-20180506.0/work/ostree-2/extra_config.json; } < /dev/null 2>&1 | /usr/bin/tee /builddir/runroot.log; exit ${PIPESTATUS[0]}'] with env {'TERM': 'vt100', 'SHELL': '/bin/bash', 'HOME': '/builddir', 'HOSTNAME': 'mock', 'PATH': '/usr/bin:/bin:/usr/sbin:/sbin', 'PROMPT_COMMAND': 'printf "\\033]0;<mock-chroot>\\007"', 'PS1': '<mock-chroot> \\s-\\v\\$ ', 'LANG': 'en_US.UTF-8'} and shell False
DEBUG util.py:296:  Unsharing. Flags: 134217728
DEBUG util.py:439:  Failed to register machine: File exists
DEBUG util.py:439:  Parent died too early
DEBUG util.py:577:  Child return code was: 1

From the error message, It looks to me that systemd-nspawn command failed when trying to register container as machine.
I see a similar upstream issue on systemd github - https://github.com/systemd/systemd/issues/2809 which is not yet resolved. If this problem continues happening frequently in future composes, maybe we can try adding option --keep-unit or --register=no as suggested in systemd upstream issue comments and see if that helps ?

dustymabe commented 5 years ago

hey @sinnykumari. do you mind opening a releng issue for this so we can link to it when these failures happen.. (i.e. so we can track how often they happen)?