cb026b4 openqa/worker: kill stuck qemu processes daily

Authored and Committed by adamwill 8 months ago
    openqa/worker: kill stuck qemu processes daily
    
    This is an awful hack to deal with
    https://github.com/os-autoinst/os-autoinst/issues/2549 while we
    try and fix it properly. This finds stuck qemu processes by
    parsing the journal messages of the workers, and kills them.
    workers stuck in the broken state should then recover on the
    next checkin with the server. I tested this manually on all the
    worker hosts and it...seemed to work, mostly. I'll keep an eye on
    things after deploying it.
    
    Signed-off-by: Adam Williamson <awilliam@redhat.com>