#36 Test failures
Closed: fixed 6 years ago Opened 7 years ago by cheimes.

33 cipher suite tests are failing in master:

$ rpm -qa nss openssl
openssl-1.0.2k-1.fc25.x86_64
nss-3.28.1-1.3.fc25.x86_64
======================================================================
FAIL: test_cipher.test_ciphers.test_3DES
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/home/heimes/dev/redhat/mod_nss/test/test_cipher.py", line 152, in test_3DES
    assert_equal_openssl("3DES")
  File "/home/heimes/dev/redhat/mod_nss/test/test_cipher.py", line 94, in assert_equal_openssl
    assert nss_list == ossl_list, '%r != %r. Difference %r' % (':'.join(nss_list), ':'.join(ossl_list), diff)
AssertionError: 'DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA' != 'AECDH-DES-CBC3-SHA:DES-CBC3-SHA:ECDH-ECDSA-DES-CBC3-SHA:ECDH-RSA-DES-CBC3-SHA:ECDHE-ECDSA-DES-CBC3-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA'. Difference set(['ECDHE-ECDSA-DES-CBC3-SHA', 'ECDH-RSA-DES-CBC3-SHA', 'ECDH-ECDSA-DES-CBC3-SHA', 'ECDHE-RSA-DES-CBC3-SHA', 'AECDH-DES-CBC3-SHA'])
...
Ran 62 tests in 1.143s

FAILED (failures=33)

Yeah, the problem is that openssl changes their configuration between each and every Fedora release so it is whack-a-mole trying to provide reasonable tests.

As upstream I try to just keep on top of the last supported Fedora release but I haven't yet tackled F-25.

I'd relax the testing but I purposely kept it wide-open to be able to catch this sort of unexpected change.

Metadata Update from @rcritten:
- Custom field type adjusted to defect

7 years ago

I diagnosed the problem with the MEDIUM test: openssl downgraded a number of ciphers from HIGH to MEDIUM in 1.0.2j

I don't want to pollute the cipher code with a million tests for specific OpenSSL versions but supporting only the latest Fedora is a bit odd too. I'm open to suggestions.

Metadata Update from @rcritten:
- Issue assigned to rcritten

7 years ago

Metadata Update from @rcritten:
- Issue close_status updated to: fixed
- Issue set to the milestone: mod_nss-1.0.15
- Issue status updated to: Closed (was: Open)

6 years ago

Login to comment on this ticket.

Metadata