b45fbe1 Reschedule Compose task in case it is stuck in 'wait' state.

Authored and Committed by jkaluza 3 years ago
    Reschedule Compose task in case it is stuck in 'wait' state.
    
    There might be infrastructure issues causing the Celery messages
    to not be delivered from frontend to backend, causing the compose
    to be stuck in the `wait` state.
    
    This commit updates and re-adds old code which has been responsible
    for rescheduling Composes stuck in the `wait` state.
    
    It is run as part of the `run_cleanup` task which is executed every
    10 minutes. If checks all the composes in the `wait` state and if
    it finds compose older than 3 days, it marks it as failed.
    
    If it finds `wait` compose older than 3 minutes, it checks if some
    backend is going to handle it and if not, it schedules it again.
    
        
file modified
+0 -50