#241 ClientSession with GSSAPI auth becomes broken after ~2 minutes
Closed: Fixed 7 years ago Opened 7 years ago by msimacek.

When I open a koji session and authenticate using GSSAPI, it will start raising AuthError after short time (between 1 and 2 minutes).
Reproducible: fairly consistently, but not always. May require multiple attempts.
Version: koji-1.11.0-1.fc25.noarch
Reproducer:

In [1]: import koji

In [2]: k=koji.ClientSession('https://koji.fedoraproject.org/kojihub')

In [3]: k.gssapi_login()
Out[3]: True

In [4]: k.getRepo('f25-build', state=koji.REPO_READY)
Out[4]: 
{'create_event': 19888495,
 'create_ts': 1481731872.53435,
 'creation_time': '2016-12-14 16:11:12.534353',
 'id': 676504,
 'state': 1}

In [5]: !date
Wed Dec 14 17:22:26 CET 2016

In [6]: !date
Wed Dec 14 17:23:52 CET 2016

In [7]: k.getRepo('f25-build', state=koji.REPO_READY)
---------------------------------------------------------------------------
AuthError                                 Traceback (most recent call last)
<ipython-input-7-b9aab7a8b1b3> in <module>()
----> 1 k.getRepo('f25-build', state=koji.REPO_READY)

/usr/lib/python2.7/site-packages/koji/__init__.pyc in __call__(self, *args, **opts)
   1941         return type(self)(self.__func, "%s.%s" % (self.__name, name))
   1942     def __call__(self, *args, **opts):
-> 1943         return self.__func(self.__name, args, opts)
   1944 
   1945 

/usr/lib/python2.7/site-packages/koji/__init__.pyc in _callMethod(self, name, args, kwargs, retry)
   2377                             tries = 0
   2378                             continue
-> 2379                     raise err
   2380                 except (SystemExit, KeyboardInterrupt):
   2381                     #(depending on the python version, these may or may not be subclasses of Exception)

AuthError: Invalid session or bad credentials

Interesting. Will try to replicate

Yes, I can replicate this myself with the following:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
#!/usr/bin/python

import koji
import time
import random

k=koji.ClientSession('https://koji.fedoraproject.org/kojihub')
k.gssapi_login()
for i in range(100):
    print k.echo('OK')
    n = random.randint(45, 180)
    print 'Sleeping for %is' % n
    time.sleep(n)

This may have nothing to do with gssapi. The hub exception is:

Traceback (most recent call last):
  File "/usr/share/koji-hub/kojixmlrpc.py", line 234, in _wrap_handler
    response = handler(environ)
  File "/usr/share/koji-hub/kojixmlrpc.py", line 277, in handle_rpc
    return self._dispatch(method, params)
  File "/usr/share/koji-hub/kojixmlrpc.py", line 301, in _dispatch
    self.check_session()
  File "/usr/share/koji-hub/kojixmlrpc.py", line 283, in check_session
    context.session = koji.auth.Session()
  File "/usr/lib/python2.7/site-packages/koji/auth.py", line 114, in __init__
    raise koji.AuthError, 'Invalid session or bad credentials'
AuthError: Invalid session or bad credentials

This exception text only happens when the session info does not match the db.

Either something is very aggressively expiring sessions in the db, or the hub is seeing sequential requests coming from different ip addresses.

Ok, I did a little digging.

The session my script created is still in the db. However, the ip address it is associated with is not the ip address it is coming from. Instead it is the private ip address of proxy10.phx2.fedoraproject.org, which is one of the two proxies that koji.fedoraproject.org currently resolves to:

% host koji.fedoraproject.org
koji.fedoraproject.org has address 209.132.181.16
koji.fedoraproject.org has address 209.132.181.15

With a long enough gap, the client will make a new connection and has a chance of switching proxies. When that happens, the session will not match.

Fedora may be adjusting some proxy settings to avoid this.

Also, I have a trial patch with some new hub options that might help

https://github.com/mikem23/koji-playground/commits/session-host-ip

Note that we have a slightly modified backport of the TrustForwardedIP in Fedora now, so that this issue is at least no longer a problem in production.

Metadata Update from @mikem:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)

7 years ago

Metadata Update from @tkopecek:
- Issue set to the milestone: 1.12

6 years ago

Login to comment on this ticket.

Metadata