koji

Clone

#249 Koji Hub attempts to verify client certificate when authtype=kerberos (and litters the log)
Closed: Fixed 4 years ago by dgregor. Opened 7 years ago by amessina.

Closed: Fixed
Reopen Issue 
[WARNING] m=sslLogin u=None p=31103 r=<IPADDR> koji.xmlrpc: Traceback (most recent call last):
File "/usr/share/koji-hub/kojixmlrpc.py", line 234, in _wrap_handler
    response = handler(environ)
File "/usr/share/koji-hub/kojixmlrpc.py", line 277, in handle_rpc
    return self._dispatch(method, params)
File "/usr/share/koji-hub/kojixmlrpc.py", line 314, in _dispatch
    ret = koji.util.call_with_argcheck(func, params, opts)
File "/usr/lib/python2.7/site-packages/koji/util.py", line 156, in call_with_argcheck
    return func(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/koji/auth.py", line 701, in sslLogin
    return context.session.sslLogin(*args, **opts)
File "/usr/lib/python2.7/site-packages/koji/auth.py", line 383, in sslLogin
    raise koji.AuthError, 'could not verify client: %s' % context.environ.get('SSL_CLIENT_VERIFY')
AuthError: could not verify client: None

Is it happening before or after krbLogin call/loglines? I'm not able to reproduce such behaviour with my setup.

@tkopecek I'm not sure how I can tell whether or not it's before or after krbLogin. I'm guessing it's before since I'm still able to login successfully using Kerberos. I am not using client certificates and I don't have one in place, which is why I think it's 'None'

raise koji.AuthError, 'could not verify client: %s' % context.environ.get('SSL_CLIENT_VERIFY')
AuthError: could not verify client: None

In same log (/var/log/error_log) should be some lines with krbLogin. Anyway, as they are only DEBUG level, they may not be seen in your setup. Can you try it with KojiDebug=On in you /etc/koji-hub/hub.conf ?

Is it possible that it is not triggered by CLI, but maybe webui client connection?

Is the auth succeeding?

The gssapi auth feature also included a change to krbLogin to have it try gssapi auth first and fall back the original krb handler.

Part of the conversation here might be slightly relevant
https://pagure.io/koji/pull-request/236#comment-15239

I think we need to find a better way for the client to autodetect which auth method will work, and certainly if kerberos auth is explicit, we should honor that.

Is the auth succeeding?

@mikem yes, the auth does succeed. Sorry for my delay in response. I'm just getting some time away from my paying job to look at this.

@tkopecek

The auth succeeds and the sslLogin occurs before the krbLogin

2017-01-13 14:45:35,788 [INFO] m=sslLogin u=None p=20424 r=<IPADDR>:57188 koji.xmlrpc: Handling method sslLogin for session None (#None)
2017-01-13 14:45:35,788 [DEBUG] m=sslLogin u=None p=20424 r=<IPADDR>:57188 koji.xmlrpc: Params: (None,)
2017-01-13 14:45:35,789 [DEBUG] m=sslLogin u=None p=20424 r=<IPADDR>:57188 koji.xmlrpc: Opts: {}
2017-01-13 14:45:35,816 [WARNING] m=sslLogin u=None p=20424 r=<IPADDR>:57188 koji.xmlrpc: Traceback (most recent call last):
File "/usr/share/koji-hub/kojixmlrpc.py", line 234, in _wrap_handler
    response = handler(environ)
File "/usr/share/koji-hub/kojixmlrpc.py", line 277, in handle_rpc
    return self._dispatch(method, params)
File "/usr/share/koji-hub/kojixmlrpc.py", line 314, in _dispatch
    ret = koji.util.call_with_argcheck(func, params, opts)
File "/usr/lib/python2.7/site-packages/koji/util.py", line 156, in call_with_argcheck
    return func(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/koji/auth.py", line 701, in sslLogin
    return context.session.sslLogin(*args, **opts)
File "/usr/lib/python2.7/site-packages/koji/auth.py", line 383, in sslLogin
    raise koji.AuthError, 'could not verify client: %s' % context.environ.get('SSL_CLIENT_VERIFY')
AuthError: could not verify client: None

2017-01-13 14:45:35,817 [DEBUG] m=sslLogin u=None p=20424 r=<IPADDR>:57188 koji.xmlrpc: Returning 1100 bytes after 0.038764 seconds
2017-01-13 14:45:35,934 [INFO] m=krbLogin u=None p=20424 r=<IPADDR>:57192 koji.xmlrpc: Handling method krbLogin for session None (#None)

PR #285

I don't think I want to force kerberos login, but then am not sure how to configure gssapi login instead. Is there something I need to change in /etc/koji.conf to use gssapi (with kerberos)? This is my /etc/koji.conf:

[koji]
server = https://example.com/kojihub
weburl = https://example.com/koji
topurl = https://example.com/kojifiles
topdir = /mnt/koji
krbservice = kojihub
authtype = kerberos
krb_rdns = False
; Hopefully temporary
;serverca = /etc/pki/tls/cert.pem

I think I am using GSSAPI login on the client side, but the hub reports krbLogin

~]$ koji --force-auth --debug --debug-xmlrpc  hello
2017-01-16 11:42:34,826 [DEBUG] koji: Opening new requests session
2017-01-16 11:42:34,827 [DEBUG] koji: Opening new requests session
url: https://example.com/kojihub/ssllogin
headers: {'Content-Length': '140', 'Content-Type': 'text/xml', 'User-Agent': 'koji/1.7'}
timeout: 60
data: "<?xml version='1.0'?>\n<methodCall>\n<methodName>sslLogin</methodName>\n<params>\n<param>\n<value><nil/></value></param>\n</params>\n</methodCall>\n"
auth: <requests_kerberos.kerberos_.HTTPKerberosAuth object at 0x7fb3a8b08550>
stream: True
2017-01-16 11:42:34,871 [DEBUG] koji: Opening new requests session
url: https://example.com/kojihub
headers: {'Content-Length': '912', 'Content-Type': 'text/xml', 'User-Agent': 'koji/1.7'}
data: "<?xml version='1.0'?>\n<methodCall>\n<methodName>krbLogin</methodName>\n<params>\n<param>\n<value><string>boICEDCCAgygAwIBBaEDAgEOogcDBQAgAAAAo4IBXWGCAVkwggFVoAMCAQWhDhsMTUVTU0lORVQu\nQ09NoiIwIKADAgEBoRkwFxsHa29qaWh1YhsMbWVzc2luZXQuY29to4IBGDCCARSgAwIBEqEDAgEB\nooIBBgSCAQLIbsf56TCuViM67OybOsHjHIVQeUMBK/SVSa7VWhj1xefPpcBwFjlep8Ad5Oe0OgYI\nRuc76qkFSCvq40Um8EYxQbavt/SUKf9mMYd2QLRfK56xM2JGp5PbTJtBwJhPiJzHBAUrjYnoOdbT\nXP+IncoMT6WEvr4iAGmMcssKtbeYgf9AnziI+MAEEMjbDCNOtTkDYwRcPWuCHPX7hP7mvpEk8s3Z\nKcmMsj/CLaOVMIcg58e/OLBGBQo40mFthRPupKSobEcFhJBq0m8PNjQd1jF6DCTypp7l4ZitBaVR\nFR5oXNjBOxfZYXGQzpJXusOnm9u+1frCRJLfXlVkzaqiLSTmiXukgZUwgZKgAwIBEqKBigSBh8PR\nxfYlgr+aD7gauYgFuYj5cToxHswGM5UYz/Rh7xwTxlKsYx1DN3v6yaJKR7Wzcc6A8SgdDfUparAE\ndnlll+GwucTR8rxwWnVeh5M/omYrONmI2nAZ3fEGOVPqMd24T6mMkT+UZkbjjfhh5RE2q/JBHyRR\n/gB3VAW5Edh54d51CkBloZHxUw==\n</string></value>\n</param>\n<param>\n<value><nil/></value></param>\n</params>\n</methodCall>\n"
timeout: 43200
stream: True
body: "<?xml version='1.0'?>\n<methodResponse>\n<params>\n<param>\n<value><array><data>\n<value><string>b1swWaADAgEFoQMCAQ+iTTBLoAMCARKiRARC+xLdvvDEurc+j0Oa0SuM5VjcewbIVbfOMjNO/ZA/\n9IBgBs7IgIT9qxGo3yPT/1SpIwT43her9r+scQgIy9nN162L\n</string></value>\n<value><string>dYGtMIGqoAMCAQWhAwIBFaOBnTCBmqADAgESooGSBIGPjBYvFGY3eX7VkL8C+wE9aeT8346Rod8e\n5wpqjsA4I8WNYrl5J6zF4uT/umowde7RD6zOa5Hkf4UhD+4Go+P8AVCwLp9ywgrqV3+rAahZrZVl\n4RHyVT7BO+bboBRUWZsLzbN5we5W/Wzl/ksnrfPFF5dMyo43qbLOrg9ibQQk1qIUp0DDwlMfhxfC\no750fj4=\n</string></value>\n<value><array><data>\n<value><string>10.77.79.3</string></value>\n<value><int>0</int></value>\n<value><string>2001:db8:c800:900:c856:7615:5fa0:8535</string></value>\n<value><int>0</int></value>\n</data></array></value>\n</data></array></value>\n</param>\n</params>\n</methodResponse>\n"
url: https://example.com/kojihub?session-id=76915&session-key=1-haP9SbPoYznXq0o0ztK&callnum=0
headers: {'Content-Length': '107', 'Content-Type': 'text/xml', 'User-Agent': 'koji/1.7'}
data: "<?xml version='1.0'?>\n<methodCall>\n<methodName>getAPIVersion</methodName>\n<params>\n</params>\n</methodCall>\n"
timeout: 43200
stream: True
body: "<?xml version='1.0'?>\n<methodResponse>\n<params>\n<param>\n<value><int>1</int></value>\n</param>\n</params>\n</methodResponse>\n"
successfully connected to hub
url: https://example.com/kojihub?session-id=76915&session-key=1-haP9SbPoYznXq0o0ztK&callnum=1
headers: {'Content-Length': '109', 'Content-Type': 'text/xml', 'User-Agent': 'koji/1.7'}
data: "<?xml version='1.0'?>\n<methodCall>\n<methodName>getLoggedInUser</methodName>\n<params>\n</params>\n</methodCall>\n"
timeout: 43200
stream: True
body: "<?xml version='1.0'?>\n<methodResponse>\n<params>\n<param>\n<value><struct>\n<member>\n<name>status</name>\n<value><int>0</int></value>\n</member>\n<member>\n<name>authtype</name>\n<value><int>1</int></value>\n</member>\n<member>\n<name>name</name>\n<value><string>user</string></value>\n</member>\n<member>\n<name>usertype</name>\n<value><int>0</int></value>\n</member>\n<member>\n<name>krb_principal</name>\n<value><string>user@EXAMPLE.COM</string></value>\n</member>\n<member>\n<name>id</name>\n<value><int>1</int></value>\n</member>\n</struct></value>\n</param>\n</params>\n</methodResponse>\n"
bonjour, user!

You are using the hub at https://example.com/kojihub
Authenticated via Kerberos principal user@EXAMPLE.COM
url: https://example.com/kojihub?session-id=76915&session-key=1-haP9SbPoYznXq0o0ztK&callnum=2
headers: {'Content-Length': '100', 'Content-Type': 'text/xml', 'User-Agent': 'koji/1.7'}
data: "<?xml version='1.0'?>\n<methodCall>\n<methodName>logout</methodName>\n<params>\n</params>\n</methodCall>\n"
timeout: 43200
stream: True
body: "<?xml version='1.0'?>\n<methodResponse>\n<params>\n<param>\n<value><nil/></value></param>\n</params>\n</methodResponse>\n"

It seems to be some problem with mutual_authentication. I'll look to it tomorrow.

I think I found my issue -- some years back I had already run into problems with WSGI and python-cryptography (and having other WSGI scripts on this host) so I had done

WSGIScriptAlias /kojihub /usr/share/koji-hub/kojixmlrpc.py
WSGIDaemonProcess kojihub processes=2 threads=15 maximum-requests=1000

<Directory "/usr/share/koji-web/scripts/">
    ...
    WSGIProcessGroup kojihub
    WSGIApplicationGroup %{GLOBAL}.
    ...
</Directory>

That resolved my previous issues, however, when I revert my configuration to use the now current default:

Alias /kojihub /usr/share/koji-hub/kojixmlrpc.py
<Directory "/usr/share/koji-web/scripts/">
    ...
    WSGIApplicationGroup %{GLOBAL}.
    ...
</Directory>

I no longer get this error. I am sorry for the noise, though it looks like it prompted some further review.

And I still see the methodName as krbLogin (which may be ok--I don't know):

data: "<?xml version='1.0'?>\n<methodCall>\n<methodName>krbLogin</methodName>

Is 'koji hello' saying that you are logged via GSSAPI or Kerberos?

 
Authenticated via Kerberos principal user@EXAMPLE.COM

And are you using mod_atuh_gssapi or just mod_auth_kerb? It seems to me like latter. (I've also added comment to PR)

For Koji, I am using neither -- neither one is configured to be used in /etc/httpd/conf.d/kojihub.conf, though I do use mod_auth_gssapi for other things on this apache instance (other <Location>)

If I'm supposed to be using mod_auth_gssapi/mod_auth_kerb on the hub, there is no indication that is now required for python-requests in apache's kojihub.conf file. This is where I'm wondering what is setting the REMOTE_USER apache environment variable.

I am using the default

Alias /kojihub /usr/share/koji-hub/kojixmlrpc.py

<Directory "/usr/share/koji-hub">
    Options ExecCGI
    SetHandler wsgi-script
    WSGIApplicationGroup %{GLOBAL}

    Require all denied
    Require ip <my_ips>
</Directory>

If we do need to use mod_auth_gssapi for the hub, should we be using the HTTP/hostname@REALM ticket or the kojihub/hostname@REALM ticket? (I'd prefer to use anything other than host/hostname@REALM) for separation.

Your 'Authenticated via Kerberos...' line indicates, that gssapi is not used at all. It just fallbacks to regular krbV, so you are not using REMOTE_USER also.

For krbV
My local working setup looks like:

<Location /koji>
    AuthType GSSAPI
    AuthName "koji GSSAPI"
    GssapiCredStore keytab:/etc/httpd/http.keytab
    GssapiLocalName On
</Location>

For krbV only it was like this:

<Location /koji>
    AuthType Kerberos
    AuthName "koji krbV"
    KrbMethodNegotiate on
    KrbMethodK5Passwd off
    Krb5Keytab /etc/httpd/http.keytab
</Location>

Principals used are documented here: https://docs.pagure.org/koji/server_howto/#setting-up-kerberos-for-authentication. Different principals could be modified/set via hub conf: https://docs.pagure.org/koji/server_howto/#id2

@tkopecek thank you for the above. I already use mod_auth_gssapi for kojiweb access and can easily add the configuration for mod_auth_gssapi for /kojihub, if that's what your saying is now needed to use GSSAPI with the hub. This hub-related GSSAPI information doesn't seem to be included in https://docs.pagure.org/koji/server_howto/#etc-httpd-conf-d-kojihub-conf or in the default kojihub.conf httpd configuration file, which is why I was struggling to figure out how REMOTE_USER was being set.

For the /kojihub credentials in the keytab, can I continue to use kojihub/example.com@EXAMPLE.COM--I'd prefer to not use hos/example.com@EXAMPLE.COM creds.

@amessina So, the GSSAPI stuff has been lacking documentation (my fault probably, sorry for that). GSSAPI is not needed, and #285 is discussing whether to make GSSAPI (or GSSAPI with kerberos fallback) an additional authmethod, so that the original issue won't appear anymore.

For configuring mod_auth_gssapi, you will need an HTTP/ service ticket.
So, in Fedora's case, we have HTTP/koji.fedoraproject.org@FEDORAPROJECT.ORG.

GSSAPI uses the /kojihub/ssllogin endpoint, so setting mod_auth_gssapi on that is enough, but to get it to work with both client certs and GSSAPI you need a slightly trickier config.
An example config for both in the same instance:

<Location /kojihub/ssllogin>
     SSLVerifyClient optional
     SSLVerifyDepth 1
     SSLOptions +StrictRequire +StdEnvVars +OptRenegotiate

     AuthType GSSAPI
     GssapiSSLonly On
     GssapiLocalName On
     AuthName "GSSAPI Single Sign On Login"
     GssapiCredStore keytab:/etc/koji-hub-http.keytab

     # This complicated ACL is to support both SSL and kerb auth at the same time
     SetEnvIfExpr "%{SSL_CLIENT_S_DN_O} == 'Fedora Project'" cert_s_o_valid
     SetEnvIfExpr "%{SSL_CLIENT_S_DN_OU} == 'Fedora User Cert'" cert_s_ou_valid
     SetEnvIfExpr "%{SSL_CLIENT_I_DN_O} == 'Fedora Project'" cert_i_o_valid
     SetEnvIfExpr "%{SSL_CLIENT_I_DN_OU} == 'Fedora Project CA'" cert_i_ou_valid

     <RequireAny>
        <RequireAll>  # For SSL cert
            Require env cert_s_o_valid
            Require env cert_s_ou_valid
            Require env cert_i_o_valid
            Require env cert_i_ou_valid
        </RequireAll>
        Require valid-user  # For GSSAPI
     </RequireAny>
</Location>

@puiterwijk thank you. This is starting to make sense.

So... in the case of GSSAPI login to the hub using your above config for 

<Location /kojihub/ssllogin>

I would no longer use the AuthPrincipal = kojihub/example.com@EXAMPLE.COM creds in hub.conf?

Note that I'm referring to the difference between using the host/example.com service vs. the kojihub/example.com service that get's set in the koji.conf, koji-gs.conf files (krbservice = kojihub).

If it makes it easier, I'm not configuring connections to Fedora's koji instance, but my own private koji/kojihub instance.

@puiterwijk, your mod_auth_gssapi configuration works like a champ. Thank you.

~]$ koji --force-auth -d --debug-xmlrpc hello
2017-01-21 11:30:56,857 [DEBUG] koji: Opening new requests session
2017-01-21 11:30:56,857 [DEBUG] koji: Opening new requests session
url: https://example.com/kojihub/ssllogin
headers: {'Content-Length': '140', 'Content-Type': 'text/xml', 'User-Agent': 'koji/1.7'}
timeout: 60
data: "<?xml version='1.0'?>\n<methodCall>\n<methodName>sslLogin</methodName>\n<params>\n<param>\n<value><nil/></value></param>\n</params>\n</methodCall>\n"
auth: <requests_kerberos.kerberos_.HTTPKerberosAuth object at 0x7f4b77fdd550>
stream: True
body: "<?xml version='1.0'?>\n<methodResponse>\n<params>\n<param>\n<value><struct>\n<member>\n<name>session-id</name>\n<value><int>77035</int></value>\n</member>\n<member>\n<name>session-key</name>\n<value><string>1-g0clgss7oTgFczh4eFO</string></value>\n</member>\n</struct></value>\n</param>\n</params>\n</methodResponse>\n"
url: https://example.com/kojihub?session-id=77035&session-key=1-g0clgss7oTgFczh4eFO&callnum=0
headers: {'Content-Length': '107', 'Content-Type': 'text/xml', 'User-Agent': 'koji/1.7'}
data: "<?xml version='1.0'?>\n<methodCall>\n<methodName>getAPIVersion</methodName>\n<params>\n</params>\n</methodCall>\n"
timeout: 43200
stream: True
body: "<?xml version='1.0'?>\n<methodResponse>\n<params>\n<param>\n<value><int>1</int></value>\n</param>\n</params>\n</methodResponse>\n"
successfully connected to hub
url: https://example.com/kojihub?session-id=77035&session-key=1-g0clgss7oTgFczh4eFO&callnum=1
headers: {'Content-Length': '109', 'Content-Type': 'text/xml', 'User-Agent': 'koji/1.7'}
data: "<?xml version='1.0'?>\n<methodCall>\n<methodName>getLoggedInUser</methodName>\n<params>\n</params>\n</methodCall>\n"
timeout: 43200
stream: True
body: "<?xml version='1.0'?>\n<methodResponse>\n<params>\n<param>\n<value><struct>\n<member>\n<name>status</name>\n<value><int>0</int></value>\n</member>\n<member>\n<name>authtype</name>\n<value><int>3</int></value>\n</member>\n<member>\n<name>name</name>\n<value><string>user</string></value>\n</member>\n<member>\n<name>usertype</name>\n<value><int>0</int></value>\n</member>\n<member>\n<name>krb_principal</name>\n<value><string>user@EXAMPLE.COM</string></value>\n</member>\n<member>\n<name>id</name>\n<value><int>1</int></value>\n</member>\n</struct></value>\n</param>\n</params>\n</methodResponse>\n"
こんにちは, user!

You are using the hub at https://example.com/kojihub
Authenticated via GSSAPI
url: https://example.com/kojihub?session-id=77035&session-key=1-g0clgss7oTgFczh4eFO&callnum=2
headers: {'Content-Length': '100', 'Content-Type': 'text/xml', 'User-Agent': 'koji/1.7'}
data: "<?xml version='1.0'?>\n<methodCall>\n<methodName>logout</methodName>\n<params>\n</params>\n</methodCall>\n"
timeout: 43200
stream: True
body: "<?xml version='1.0'?>\n<methodResponse>\n<params>\n<param>\n<value><nil/></value></param>\n</params>\n</methodResponse>\n"

@puiterwijk Would you mind extending docs about GSSAPI setup? httpd.conf derived from Fedora setup would be also helpful. (If not, I can do it, but probably with less insight).

@tkopecek I will look at writing some docs and sending a PR soon.

In the meantime, I can give you the full Fedora production httpd config for the hub: https://infrastructure.fedoraproject.org/cgit/ansible.git/tree/roles/koji_hub/templates/kojihub.conf.j2

Closing this issue. Remaining documentation work will be included in https://pagure.io/koji/issue/1400

Metadata Update from @dgregor:
- Custom field Size adjusted to None
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)
4 years ago

Login to comment on this ticket.

Metadata
None
None
None
None
None
None
None
Powered by Pagure 5.13.3
Documentation • File an Issue • About • SSH Hostkey/Fingerprint
© Red Hat, Inc. and others.