#47 inventory: Redirect qemu stderr to prevent ansible-playbook hang when run from a service
Merged by astepano. Opened by merlinm.
Unknown source master

Download 47.patch

When ansible-playbook is started from a service (such as cron or Jenkins)
and uses a test subject launched by the dynamic inventory provider
standard-inventory-qcow2 script, ansible-playbook hangs as if it were
still waiting for the dynamic inventory to be gathered. The hang does not
occur when run from the command line.

The cause of the hang was identified to be the stderr file descriptor passed
to qemu when it is started. This commit redirects stderr to /dev/null.

The following script can be used to demonstrate the current problem and the fix. It runs normally from the command line, but will hang if started from cron, batch, etc. (Be sure to manually kill the lingering qemu process after the hang, and clean up the temporary directory.)

#!/bin/bash
mkdir /tmp/workspace$$
cd /tmp/workspace$$
cat - > ping.yml <<EOF
---
- hosts: localhost
  tasks:
  - ping:
EOF
curl -Lo atomic.qcow2 https://ftp-stud.hs-esslingen.de/pub/Mirrors/alt.fedoraproject.org/atomic/stable/Fedora-Atomic-26-20170707.1/CloudImages/x86_64/images/Fedora-Atomic-26-20170707.1.x86_64.qcow2
curl -Lo standard-inventory-qcow2 https://pagure.io/fork/merlinm/standard-test-roles/raw/fix-hang-when-run-from-service/f/inventory/standard-inventory-qcow2
chmod +x standard-inventory-qcow2
export TEST_SUBJECTS=$PWD/atomic.qcow2
export ANSIBLE_INVENTORY=$PWD/standard-inventory-qcow2
ansible-playbook ping.yml
export ANSIBLE_INVENTORY=/usr/share/ansible/inventory/standard-inventory-qcow2
# this will hang if run from cron, batch, etc.
ansible-playbook ping.yml

I don't think we should hide information that makes debugging harder later. Can we redirect stdout and stderr to the qemu the machine log file? ... that is the one opened on the line above.

I completely agree. I mentioned in a comment on PR #52 (which reported this hang after I already raised this PR to patch it), I have plans to make a future enhancement to the script to redirect that output to a logfile (along with other diagnostic/trace information), but I'm waiting until after the PR#39 work is completed to avoid extensive dependencies. In the mean time, however, this is a quick fix to workaround the background hang issue--which is a show-stopper for automated testing.

rebased onto 7fde356416e09bf55fafafe8decd52fb8f7cd828

1 new commit added

  • Redirect stderr to qemu machine log instead of discarding it.

rebased onto ca38c5c229d9af3929b372eeb0219a9cde9b69cb

@stefw My last commit addresses your suggestion to redirect stdout and stderr to the qemu machine log file rather than /dev/null.

Pull-Request has been merged by astepano

Metadata