PR#1358: rollback errors in multiCall - koji

koji

#1358 rollback errors in multiCall

Merged 4 years ago by mikem. Opened 5 years ago by mikem.

mikem/koji multicall-rollback into master

document multicall

Mike McLean • 4 years ago

7f64d78

rollback errors in multiCall

Mike McLean • 4 years ago

a8fb6c9

docs/source/writing_koji_code.rst

file modified

+56

		`@@ -174,6 +174,62 @@`
		`function. Here you can define the string(s) to display when Koji`
		`receives status on a task. That is the return value.`

		`+ Using multiCall`
		`+ ~~~~~~~~~~~~~~~`
		`+`
		`+ Koji supports a multicall feature where many calls are passed to the`
		`+ server wrapped as a single call. This can reduce the overhead when a`
		`+ large number of related calls need to be made.`
		`+`
		+ The ``ClientSession`` class provides support for this and there are several
		`+ examples in the existing client code. Some examples in the cli include:`
		+ ``edit-host``, ``add-pkg``, ``disable-host``, and ``list-hosts``.
		`+`
		+ To use the feature, you first set the ``multicall`` attribute of the session
		+ to ``True``. Once this is done, the session will not immediately process
		`+ further calls but will instead store their parameters for later. To tell the`
		+ session to process them, use the ``multiCall()`` method (note the
		`+ capitalization).`
		`+`
		+ The ``multiCall()`` method returns a list of results, one for each call
		`+ in the multicall. Each result with either be:`
		`+`
		`+ 1. the result of the call wrapped in a singleton list`
		`+ 2. a dictionary representing the error raised by the call`
		`+`
		`+ Here is a simple example from the koji-tools package:`
		`+`
		`+ ::`
		`+`
		`+ session.multicall = True`
		`+ for host in hosts:`
		`+ session.listChannels(hostID=host['id'])`
		`+ for host, [channels] in zip(hosts, session.multiCall(strict=True)):`
		`+ host['channels'] = channels`
		`+`
		`+ Note that when using multicall for informational calls, it is important`
		`+ to keep track of which result is which. Here we use the existing hosts`
		+ list as a unifying index. Python's ``zip`` function is useful here.
		`+ Also note the unpacking of the singletons.`
		`+`
		+ The ``multiCall()`` method supports a few options. Here is its signature:
		`+`
		`+ ::`
		`+`
		`+ multiCall(strict=False, batch=None):`
		`+`
		`+ If the strict option is set to True, then this method will raise the`
		`+ first error it encounters, if any.`
		`+`
		`+ If the batch option is set to a number greater than zero, the calls`
		`+ will be spread across multiple multicall batches of at most this`
		`+ number.`
		`+`
		`+ The hub processes multicalls in a single database transaction. Note that if`
		+ the ``batch`` option is used, then each batch is a separate multicall in the
		`+ api and therefore a separate transaction.`
		`+`
		`+`
		`Koji-Hub`
		`--------`

hub/kojixmlrpc.py

file modified

+4 -1

		`@@ -329,11 +329,14 @@`
		`results and errors, and return those as a list."""`
		`results = []`
		`for call in calls:`
		`+ savepoint = kojihub.Savepoint('multiCall_loop')`
		`try:`
		`result = self._dispatch(call['methodName'], call['params'])`
		`except Fault as fault:`
		`+ savepoint.rollback()`
		`results.append({'faultCode': fault.faultCode, 'faultString': fault.faultString})`
		`- except:`
		`+ except Exception:`
		`+ savepoint.rollback()`
		`# transform unknown exceptions into XML-RPC Faults`
		`# don't create a reference to full traceback since this creates`
		`# a circular reference.`

mikem commented 5 years ago

Fixes: https://pagure.io/koji/issue/1357

mikem commented 5 years ago

@tkopecek

mikem commented 5 years ago

h/t @ahills

tkopecek commented 5 years ago

Also there is no clear understanding of how commits should work for multicall. Do we expect, that each call has its own transaction (as it could be naive expectation), or that all calls are committed at once (as it is in reality)? It probably does matter only for pre/postCommit callbacks and possibly for some intended "atomic" calls. It is probably just matter of docs to be more clear on that, then of changing the code.

mikem commented 5 years ago