#1358 rollback errors in multiCall
Merged 3 years ago by mikem. Opened 3 years ago by mikem.
mikem/koji multicall-rollback  into  master

@@ -174,6 +174,62 @@ 

  function. Here you can define the string(s) to display when Koji

  receives status on a task. That is the return value.


+ Using multiCall

+ ~~~~~~~~~~~~~~~


+ Koji supports a multicall feature where many calls are passed to the

+ server wrapped as a single call. This can reduce the overhead when a

+ large number of related calls need to be made.


+ The ``ClientSession`` class provides support for this and there are several

+ examples in the existing client code. Some examples in the cli include:

+ ``edit-host``, ``add-pkg``, ``disable-host``, and ``list-hosts``.


+ To use the feature, you first set the ``multicall`` attribute of the session

+ to ``True``. Once this is done, the session will not immediately process

+ further calls but will instead store their parameters for later. To tell the

+ session to process them, use the ``multiCall()`` method (note the

+ capitalization).


+ The ``multiCall()`` method returns a list of results, one for each call

+ in the multicall. Each result with either be:


+ 1. the result of the call wrapped in a singleton list

+ 2. a dictionary representing the error raised by the call


+ Here is a simple example from the koji-tools package:


+ ::


+     session.multicall = True

+     for host in hosts:

+         session.listChannels(hostID=host['id'])

+     for host, [channels] in zip(hosts, session.multiCall(strict=True)):

+         host['channels'] = channels


+ Note that when using multicall for informational calls, it is important

+ to keep track of which result is which. Here we use the existing hosts

+ list as a unifying index. Python's ``zip`` function is useful here.

+ Also note the unpacking of the singletons.


+ The ``multiCall()`` method supports a few options. Here is its signature:


+ ::


+     multiCall(strict=False, batch=None):


+ If the strict option is set to True, then this method will raise the

+ first error it encounters, if any.


+ If the batch option is set to a number greater than zero, the calls

+ will be spread across multiple multicall batches of at most this

+ number.


+ The hub processes multicalls in a *single database transaction*. Note that if

+ the ``batch`` option is used, then each batch is a separate multicall in the

+ api and therefore a separate transaction.






file modified
+4 -1
@@ -329,11 +329,14 @@ 

          results and errors, and return those as a list."""

          results = []

          for call in calls:

+             savepoint = kojihub.Savepoint('multiCall_loop')


                  result = self._dispatch(call['methodName'], call['params'])

              except Fault as fault:

+                 savepoint.rollback()

                  results.append({'faultCode': fault.faultCode, 'faultString': fault.faultString})

-             except:

+             except Exception:

+                 savepoint.rollback()

                  # transform unknown exceptions into XML-RPC Faults

                  # don't create a reference to full traceback since this creates

                  # a circular reference.

Also there is no clear understanding of how commits should work for multicall. Do we expect, that each call has its own transaction (as it could be naive expectation), or that all calls are committed at once (as it is in reality)? It probably does matter only for pre/postCommit callbacks and possibly for some intended "atomic" calls. It is probably just matter of docs to be more clear on that, then of changing the code.

each call has its own transaction (as it could be naive expectation)

I'm not sure that's the expectation. I've spoken to a number of people that expected multicall to be single transaction, as it is.

Agreed it should be documented

1 new commit added

  • document multicall
3 years ago

pretty please pagure-ci rebuild

3 years ago

rebased onto a8fb6c9

3 years ago

Commit 7fb56ee fixes this pull-request

Pull-Request has been merged by mikem

3 years ago