#6796 WSGI fails with recursion error in GSSAPI
Closed: fixed 6 years ago Opened 7 years ago by cheimes.

ipa ping fails with either one of this error messages all the time. Internal server error is the recursion issue.

$ ipa ping
ipa: ERROR: cannot connect to 'https://host/ipa/json': Internal Server Error
$ ipa ping
ipa: ERROR: Major (851968): Unspecified GSS failure.  Minor code may provide more information, Minor (69206038): Invalid argument

The issue is similar to another bug I filed a while ago https://github.com/pythongssapi/python-gssapi/issues/111 . Contrary to the other issue, I haven't been able to recover the server from the problem. Neither kinit nor ipactl restart nor reboot got rid of the problem. System is running in permissive mode.

  • python-gssapi-1.2.0-2.fc25.x86_64
  • cyrus-sasl-2.1.26-26.2.simo5.fc25.x86_64
  • freeipa-server-4.5.90.dev201703211543+git3de0970-0.fc25.x86_64

Apache error log

mod_wsgi (pid=2897): Exception occurred processing WSGI script '/usr/share/ipa/wsgi.py'.
Traceback (most recent call last):
  File "/usr/share/ipa/wsgi.py", line 51, in application
    return api.Backend.wsgi_dispatch(environ, start_response)
  File "/usr/lib/python2.7/site-packages/ipaserver/rpcserver.py", line 262, in __call__
    return self.route(environ, start_response)
  File "/usr/lib/python2.7/site-packages/ipaserver/rpcserver.py", line 274, in route
    return app(environ, start_response)
  File "/usr/lib/python2.7/site-packages/ipaserver/rpcserver.py", line 468, in __call__
    response = super(jsonserver, self).__call__(environ, start_response)
  File "/usr/lib/python2.7/site-packages/ipaserver/rpcserver.py", line 648, in __call__
    self.create_context(ccache=user_ccache)
  File "/usr/lib/python2.7/site-packages/ipalib/backend.py", line 125, in create_context
    time_limit=None)
  File "/usr/lib/python2.7/site-packages/ipalib/backend.py", line 66, in connect
    conn = self.create_connection(*args, **kw)
  File "/usr/lib/python2.7/site-packages/ipaserver/plugins/ldap2.py", line 205, in create_connection
    principal = krb_utils.get_principal(ccache_name=ccache)
  File "/usr/lib/python2.7/site-packages/ipalib/krb_utils.py", line 168, in get_principal
    creds = get_credentials(ccache_name=ccache_name)
  File "/usr/lib/python2.7/site-packages/ipalib/krb_utils.py", line 147, in get_credentials
    return gssapi.Credentials(usage='initiate', name=name, store=store)
  File "/usr/lib64/python2.7/site-packages/gssapi/creds.py", line 64, in __new__
    store=store)
  File "/usr/lib64/python2.7/site-packages/gssapi/creds.py", line 148, in acquire
    usage)
  File "gssapi/raw/ext_cred_store.pyx", line 182, in gssapi.raw.ext_cred_store.acquire_cred_from (gssapi/raw/ext_cred_store.c:1886)
  File "gssapi/raw/misc.pyx", line 216, in gssapi.raw.misc.GSSErrorRegistry.__call__ (gssapi/raw/misc.c:3057)
  File "gssapi/raw/misc.pyx", line 272, in gssapi.raw.misc.GSSError.__init__ (gssapi/raw/misc.c:3531)
  File "gssapi/raw/misc.pyx", line 325, in gssapi.raw.misc.GSSError.gen_message (gssapi/raw/misc.c:4484)
  File "gssapi/raw/misc.pyx", line 295, in gssapi.raw.misc.GSSError.get_all_statuses (gssapi/raw/misc.c:3819)
  File "gssapi/raw/misc.pyx", line 174, in gssapi.raw.misc._display_status (gssapi/raw/misc.c:1960)
  File "gssapi/raw/misc.pyx", line 216, in gssapi.raw.misc.GSSErrorRegistry.__call__ (gssapi/raw/misc.c:3057)

  File "gssapi/raw/misc.pyx", line 295, in gssapi.raw.misc.GSSError.get_all_statuses (gssapi/raw/misc.c:3819)
  File "gssapi/raw/misc.pyx", line 174, in gssapi.raw.misc._display_status (gssapi/raw/misc.c:1960)
  File "gssapi/raw/misc.pyx", line 216, in gssapi.raw.misc.GSSErrorRegistry.__call__ (gssapi/raw/misc.c:3057)
  File "gssapi/raw/misc.pyx", line 272, in gssapi.raw.misc.GSSError.__init__ (gssapi/raw/misc.c:3531)
  File "gssapi/raw/misc.pyx", line 325, in gssapi.raw.misc.GSSError.gen_message (gssapi/raw/misc.c:4484)
  File "gssapi/raw/misc.pyx", line 295, in gssapi.raw.misc.GSSError.get_all_statuses (gssapi/raw/misc.c:3819)
  File "gssapi/raw/misc.pyx", line 174, in gssapi.raw.misc._display_status (gssapi/raw/misc.c:1960)
  File "gssapi/raw/misc.pyx", line 214, in gssapi.raw.misc.GSSErrorRegistry.__call__ (gssapi/raw/misc.c:2986)
  File "gssapi/raw/misc.pyx", line 209, in gssapi.raw.misc.GSSErrorRegistry.__find_error (gssapi/raw/misc.c:2761)
  File "gssapi/raw/misc.pyx", line 202, in gssapi.raw.misc.GSSErrorRegistry.__get_registry (gssapi/raw/misc.c:2458)
RuntimeError: maximum recursion depth exceeded in cmp

KDC log

Mar 22 09:35:32 host krb5kdc[10094](info): AS_REQ (6 etypes {18 17 16 23 25 26}) 10.34.58.207: ISSUE: authtime 1490171732, etypes {rep=18 tkt=18 ses=18}, admin@REALM for krbtgt/REALM@REALM
Mar 22 09:35:32 host krb5kdc[10094](info): closing down fd 11
Mar 22 09:35:41 host krb5kdc[10094](info): TGS_REQ (6 etypes {18 17 16 23 25 26}) 10.34.58.207: ISSUE: authtime 1490171732, etypes {rep=18 tkt=18 ses=18}, admin@REALM for HTTP/host@REALM
Mar 22 09:35:41 host krb5kdc[10094](info): closing down fd 11
Mar 22 09:35:41 host krb5kdc[10094](info): AS_REQ (6 etypes {18 17 16 23 25 26}) 10.34.58.207: NEEDED_PREAUTH: HTTP/host@REALM for krbtgt/REALM@REALM, Additional pre-authentication required
Mar 22 09:35:41 host krb5kdc[10094](info): closing down fd 11
Mar 22 09:35:41 host krb5kdc[10093](info): AS_REQ (6 etypes {18 17 16 23 25 26}) 10.34.58.207: ISSUE: authtime 1490171741, etypes {rep=18 tkt=18 ses=18}, HTTP/host@REALM for krbtgt/REALM@REALM
Mar 22 09:35:41 host krb5kdc[10093](info): closing down fd 11

Triage comments:
- python-gssapi bug https://github.com/pythongssapi/python-gssapi/pull/112
- it should not be possible to run into infinite recursions - ever
- +1 having recursive calls without clear termination condition is just bad practice
4.5.1

Metadata Update from @pvoborni:
- Issue priority set to: critical
- Issue set to the milestone: FreeIPA 4.5.1
- Issue tagged with: bug

7 years ago

Metadata Update from @pvoborni:
- Custom field rhbz adjusted to https://bugzilla.redhat.com/show_bug.cgi?id=1438016

7 years ago

Metadata Update from @pvoborni:
- Custom field rhbz adjusted to https://bugzilla.redhat.com/show_bug.cgi?id=1438016

7 years ago

The python-gssapi part is fixed in 1.2.0-5 build.

Metadata Update from @mbasti:
- Issue set to the milestone: FreeIPA 4.5.2 (was: FreeIPA 4.5.1)

6 years ago

FreeIPA 4.5.1 has been released, moving to FreeIPA 4.5.2 milestone

Metadata Update from @stlaz:
- Issue assigned to stlaz

6 years ago

master:

  • 81a808c Avoid possible endless recursion in RPC call
  • 79d1752 rpc: preparations for recursion fix
  • e1f8684 rpc: avoid possible recursion in create_connection

ipa-4-5:

  • a5b413b Avoid possible endless recursion in RPC call
  • d8aab38 rpc: preparations for recursion fix
  • cb6c93d rpc: avoid possible recursion in create_connection

Metadata Update from @mbasti:
- Issue close_status updated to: fixed
- Issue status updated to: Closed (was: Open)

6 years ago

master:

  • 2485c33 Bump version of python-gssapi

ipa-4-5:

  • 15d5ddd Bump version of python-gssapi

Login to comment on this ticket.

Metadata