ca9d3a2 Add celery task router

9 files Authored by qwan 4 years ago, Committed by mprahl 4 years ago,
    Add celery task router
    
    Add route_task function to route celery tasks to different queues.
    If we can figure out what the module build is a task ran for by
    checking the task arguments, then we route this task to a queue
    named:
    
        "mbs-{}".format(module_build_id % num_workers)
    
    "num_workers" has default value of 1, and can be changed in
    backend_config.py. If module build id can't be figured out, task will
    be routed to the default queue which is named "mbs-default".
    
    While setting up the workers, the number of workers should match with
    "num_workers" in config, and each worker will listen on two queues:
    
        1. mbs-default
        2. mbs-{number} # for example, the first worker listens on "mbs-0"
    
    By this design, all tasks for a particular module build will be routed
    to the same queue and run on the same worker serially.