#4649 IPA Upgrade: PySSL_sslwrap SIGSEGV
Closed: Fixed None Opened 7 years ago by mbasti.

We don't have reproducer. It happens rarely.

(We found this during DNSSEC work, but we don't use PySSL in DNSSEC)

Oct 19 01:37:45 example.com ipactl[14145]: Starting ipa-dnskeysyncd Service

Oct 19 01:37:51 example.com systemd-coredump[14049]: Process 13333 (ipa-upgradeconf) of user 0 dumped core.                                                                     
                                                                       Stack trace of thread 13333:
                                                                       #0  0x00007f0acf3e5de4 PySSL_sslwrap (_ssl.so)
                                                                       #1  0x00007f0ad89ddade PyEval_EvalFrameEx (libpython2.7.so.1.0)
                                                                       #2  0x00007f0ad89de490 PyEval_EvalCodeEx (libpython2.7.so.1.0)
                                                                       #3  0x00007f0ad896a7bd function_call (libpython2.7.so.1.0)
                                                                       #4  0x00007f0ad89458d3 PyObject_Call (libpython2.7.so.1.0)
                                                                       #5  0x00007f0ad89547e5 instancemethod_call (libpython2.7.so.1.0)
                                                                       #6  0x00007f0ad89458d3 PyObject_Call (libpython2.7.so.1.0)
                                                                       #7  0x00007f0ad899c927 slot_tp_init (libpython2.7.so.1.0)
                                                                       #8  0x00007f0ad899b65f type_call (libpython2.7.so.1.0)
                                                                       #9  0x00007f0ad89458d3 PyObject_Call (libpython2.7.so.1.0)
                                                                       #10 0x00007f0ad89db615 PyEval_EvalFrameEx (libpython2.7.so.1.0)
                                                                       #11 0x00007f0ad89de490 PyEval_EvalCodeEx (libpython2.7.so.1.0)
                                                                       #12 0x00007f0ad89dcb74 PyEval_EvalFrameEx (libpython2.7.so.1.0)
                                                                       #13 0x00007f0ad89dcc76 PyEval_EvalFrameEx (libpython2.7.so.1.0)
                                                                       #14 0x00007f0ad89dcc76 PyEval_EvalFrameEx (libpython2.7.so.1.0)
                                                                       #15 0x00007f0ad89de490 PyEval_EvalCodeEx (libpython2.7.so.1.0)
                                                                       #16 0x00007f0ad89dcb74 PyEval_EvalFrameEx (libpython2.7.so.1.0)
                                                                       Stack trace of thread 13333:
                                                                       #0  0x00007f0acf3e5de4 PySSL_sslwrap (_ssl.so)
                                                                       #1  0x00007f0ad89ddade PyEval_EvalFrameEx (libpython2.7.so.1.0)
                                                                       #2  0x00007f0ad89de490 PyEval_EvalCodeEx (libpython2.7.so.1.0)
                                                                       #3  0x00007f0ad896a7bd function_call (libpython2.7.so.1.0)
                                                                       #4  0x00007f0ad89458d3 PyObject_Call (libpython2.7.so.1.0)
                                                                       #5  0x00007f0ad89547e5 instancemethod_call (libpython2.7.so.1.0)
                                                                       #6  0x00007f0ad89458d3 PyObject_Call (libpython2.7.so.1.0)
                                                                       #7  0x00007f0ad899c927 slot_tp_init (libpython2.7.so.1.0)
                                                                       #8  0x00007f0ad899b65f type_call (libpython2.7.so.1.0)
                                                                       #9  0x00007f0ad89458d3 PyObject_Call (libpython2.7.so.1.0)
                                                                       #10 0x00007f0ad89db615 PyEval_EvalFrameEx (libpython2.7.so.1.0)
                                                                       #11 0x00007f0ad89de490 PyEval_EvalCodeEx (libpython2.7.so.1.0)
                                                                       #12 0x00007f0ad89dcb74 PyEval_EvalFrameEx (libpython2.7.so.1.0)
                                                                       #13 0x00007f0ad89dcc76 PyEval_EvalFrameEx (libpython2.7.so.1.0)
                                                                       #14 0x00007f0ad89dcc76 PyEval_EvalFrameEx (libpython2.7.so.1.0)
                                                                       #15 0x00007f0ad89de490 PyEval_EvalCodeEx (libpython2.7.so.1.0)
                                                                       #16 0x00007f0ad89dcb74 PyEval_EvalFrameEx (libpython2.7.so.1.0)
                                                                       #17 0x00007f0ad89de490 PyEval_EvalCodeEx (libpython2.7.so.1.0)
                                                                       #18 0x00007f0ad89dcb74 PyEval_EvalFrameEx (libpython2.7.so.1.0)
                                                                       #19 0x00007f0ad89dcc76 PyEval_EvalFrameEx (libpython2.7.so.1.0)
                                                                       #20 0x00007f0ad89de490 PyEval_EvalCodeEx (libpython2.7.so.1.0)
                                                                       #21 0x00007f0ad89dcb74 PyEval_EvalFrameEx (libpython2.7.so.1.0)
                                                                       #22 0x00007f0ad89dcc76 PyEval_EvalFrameEx (libpython2.7.so.1.0)
                                                                       #23 0x00007f0ad89de490 PyEval_EvalCodeEx (libpython2.7.so.1.0)
                                                                       #24 0x00007f0ad89dcb74 PyEval_EvalFrameEx (libpython2.7.so.1.0)
                                                                       #25 0x00007f0ad89de490 PyEval_EvalCodeEx (libpython2.7.so.1.0)
                                                                       #26 0x00007f0ad89dcb74 PyEval_EvalFrameEx (libpython2.7.so.1.0)
                                                                       #27 0x00007f0ad89dcc76 PyEval_EvalFrameEx (libpython2.7.so.1.0)
                                                                       #28 0x00007f0ad89de490 PyEval_EvalCodeEx (libpython2.7.so.1.0)
                                                                       #29 0x00007f0ad89dcb74 PyEval_EvalFrameEx (libpython2.7.so.1.0)
                                                                       #30 0x00007f0ad89de490 PyEval_EvalCodeEx (libpython2.7.so.1.0)
                                                                       #31 0x00007f0ad89dcb74 PyEval_EvalFrameEx (libpython2.7.so.1.0)
                                                                       #32 0x00007f0ad89dcc76 PyEval_EvalFrameEx (libpython2.7.so.1.0)
                                                                       #33 0x00007f0ad89de490 PyEval_EvalCodeEx (libpython2.7.so.1.0)
                                                                       #34 0x00007f0ad89dcb74 PyEval_EvalFrameEx (libpython2.7.so.1.0)
                                                                       #35 0x00007f0ad89de490 PyEval_EvalCodeEx (libpython2.7.so.1.0)
                                                                       #36 0x00007f0ad89de589 PyEval_EvalCode (libpython2.7.so.1.0)
                                                                       #37 0x00007f0ad89f793f run_mod (libpython2.7.so.1.0)
                                                                       #38 0x00007f0ad89f8b62 PyRun_FileExFlags (libpython2.7.so.1.0)
                                                                       #39 0x00007f0ad89f9d77 PyRun_SimpleFileExFlags (libpython2.7.so.1.0)
                                                                       #40 0x00007f0ad8a0b30a Py_Main (libpython2.7.so.1.0)
                                                                       #41 0x00007f0ad7c32fe0 __libc_start_main (libc.so.6)
                                                                       #42 0x000000000040071e _start (python2.7)
Oct 19 01:37:53 example.com /usr/share/ipa/ipadnssecd/ipa-ods-exporter.py[14535]: Kerberos principal: ipa-ods-exporter/example.com

Probably this is reason why we are getting segmentation fault.

There is null pointer dereference inside python standard library

#0  0x00007f3c4a66dde4 in newPySSLObject (ciphers=0x7f3c4544eeb4 "DEFAULT:!aNULL:!eNULL:!LOW:!EXPORT:!SSLv2", cacerts_file=<optimized out>, proto_version=PY_SSL_VERSION_SSL23, 
    certreq=<optimized out>, socket_type=<optimized out>, cert_file=0x0, key_file=0x0, Sock=0x7f3c3fbafc30) at /usr/src/debug/Python-2.7.8/Modules/_ssl.c:317
317         self->ctx->options &= ~(SSL_OP_NO_SSLv2 | SSL_OP_NO_SSLv3);

(gdb) p self->ctx
$1 = (SSL_CTX *) 0x0

bt
#0  0x00007f3c4a66dde4 in newPySSLObject (ciphers=0x7f3c4544eeb4 "DEFAULT:!aNULL:!eNULL:!LOW:!EXPORT:!SSLv2", cacerts_file=<optimized out>, proto_version=PY_SSL_VERSION_SSL23, 
    certreq=<optimized out>, socket_type=<optimized out>, cert_file=0x0, key_file=0x0, Sock=0x7f3c3fbafc30) at /usr/src/debug/Python-2.7.8/Modules/_ssl.c:317
#1  PySSL_sslwrap (self=<optimized out>, args=<optimized out>) at /usr/src/debug/Python-2.7.8/Modules/_ssl.c:455
#2  0x00007f3c53c65ade in call_function (oparg=<optimized out>, pp_stack=<optimized out>) at /usr/src/debug/Python-2.7.8/Python/ceval.c:4110
#3  PyEval_EvalFrameEx (f=<unknown at remote 0x2e10960>, throwflag=1085658968) at /usr/src/debug/Python-2.7.8/Python/ceval.c:2753
#4  0x00007f3c53c66490 in PyEval_EvalCodeEx (co=0x7f3c45436730, globals=0x0, locals=< at remote 0x23e20a0>, locals@entry=0x0, args=0x2, 
....

This is called at ipa-upgradeconfig:1457 ca.restart(...), where the check if CA is up is, which uses https (ipapython.dogtag.ca_status).

    status, reason, headers, body = unauthenticated_https_request(
        ca_host, ca_port, '/ca/admin/ca/getStatus')

tail of ipaupgrade.log:

2014-10-23T23:42:35Z DEBUG stderr=
2014-10-23T23:42:35Z DEBUG wait_for_open_ports: localhost [8080, 8443] timeout 300
2014-10-23T23:42:38Z DEBUG Waiting until the CA is running
2014-10-23T23:42:38Z DEBUG request 'https://vm.example.com:443/ca/admin/ca/getStatus'
2014-10-23T23:42:38Z DEBUG request body ''

Fortunately it is last step of upgrading process, so ipa was upgraded correctly

Thanks for assessment. We will need the Bugzilla filed, issue fixed&Requires bumped...

This hits only F21.

Problem fixed in python-2.7.8-5.fc21

No need to create downstream Bugzilla, this is fixed.

Metadata Update from @mbasti:
- Issue assigned to mbasti
- Issue set to the milestone: FreeIPA 4.1.1

5 years ago

Login to comment on this ticket.

Metadata