fm-orchestrator

Clone
Source Code
GIT

#1539 Force the Celery workers to run only one task at a time
Merged 4 years ago by mprahl. Opened 4 years ago by mprahl.
ensure-serial  into  v3

file modified
+3 -1 
@@ -53,8 +53,10 @@ 
 

  # celery_broker_url -> broker_url
 

  celery_configs = {
 

      name[7:]: getattr(conf, name)
 

-     for name in dir(conf) if name.startswith('celery_')
 

+     for name in dir(conf) if name.startswith("celery_")
 

  }
 

+ # Only allow a single process so that tasks are always serial per worker
 

+ celery_configs["worker_concurrency"] = 1
 

  celery_app.conf.update(**celery_configs)

file modified
+7 
@@ -656,6 +656,13 @@ 
 

              "desc": "The minrate configuration on a DNF repo. This configuration will cause DNF to "
 

                      "timeout loading a repo if the download speed is below minrate for the "
 

                      "duration of the timeout."
 

+         },
 

+         "celery_worker_prefetch_multiplier": {
 

+             "type": int,
 

+             "default": 1,
 

+             "desc": "This defaults to 1 so that the worker doesn't fetch more messages than it can "
 

+                     "handle at a time. This so that general tasks aren't starved when running "
 

+                     "a long handler.",
 

          }
 

      }

no initial comment

Why not add CELERY_WORKER_CONCURRENCY to config.py?

The config files have been reverted to original conf/config.py. Could you please rebase this PR?

Why not add CELERY_WORKER_CONCURRENCY to config.py?

Because we won't allow concurrently for workers, so it's not configurable, otherwise the design of multi-workers will break.

rebased onto ad49a26
4 years ago

@cqi @qwan this is now rebased.

:thumbsup:

Build #685 failed (commit: 9703c8c1019c121fe537cde4c71cc2b978c8c62a).
Rebase or make new commits to rebuild.

2 new commits added

  • Force the Celery workers to run only one task at a time
  • Use double quotes to match the coding standard
4 years ago

Pull-Request has been merged by mprahl
4 years ago
Changes Summary 2
+3 -1
file changed
module_build_service/__init__.py
+7 -0
file changed
module_build_service/config.py
Powered by Pagure 5.13.3
Documentation • File an Issue • About • SSH Hostkey/Fingerprint
© Red Hat, Inc. and others.