#3562 Bug in assert_deepequal causes unit test failure
Closed: Fixed None Opened 10 years ago by akrivoka.

assert_deepequal should not raise an exception in the following example:

[akrivoka@vm-060 tests]$ python
Python 2.7.3 (default, Aug  9 2012, 17:23:57) 
[GCC 4.7.1 20120720 (Red Hat 4.7.1-5)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import util
>>> import ipapython
>>> expected = {'count': 4,
...  'result': [{'dn': ipapython.dn.DN('idnsname=dnszone.test,cn=dns,dc=ipa,dc=test'),
...              'idnsname': [u'@'],
...              'nsrecord': (u'ns1.dnszone.test.',)},
...             {'dn': ipapython.dn.DN('idnsname=_kerberos,idnsname=dnszone.test,cn=dns,dc=ipa,dc=test'),
...              'idnsname': [u'_kerberos'],
...              'txtrecord': [u'IPA.TEST']},
...             {'arecord': [u'1.2.3.4'],
...              'dn': ipapython.dn.DN('idnsname=ns1,idnsname=dnszone.test,cn=dns,dc=ipa,dc=test'),
...              'idnsname': [u'ns1']},
...             {'arecord': [u'127.0.0.1'],
...              'dn': ipapython.dn.DN('idnsname=testdnsres,idnsname=dnszone.test,cn=dns,dc=ipa,dc=test'),
...              'idnsname': [u'testdnsres']}],
...  'summary': None,
...  'truncated': False}
>>> got = {'count': 4,
...  'result': ({'dn': u'idnsname=dnszone.test,cn=dns,dc=ipa,dc=test',
...              'idnsname': (u'@',),
...              'nsrecord': (u'ns1.dnszone.test.',)},
...             {'dn': u'idnsname=_kerberos,idnsname=dnszone.test,cn=dns,dc=ipa,dc=test',
...              'idnsname': (u'_kerberos',),
...              'txtrecord': (u'IPA.TEST',)},
...             {'arecord': (u'1.2.3.4',),
...              'dn': u'idnsname=ns1,idnsname=dnszone.test,cn=dns,dc=ipa,dc=test',
...              'idnsname': (u'ns1',)},
...             {'arecord': (u'127.0.0.1',),
...              'dn': u'idnsname=testdnsres,idnsname=dnszone.test,cn=dns,dc=ipa,dc=test',
...              'idnsname': (u'testdnsres',)}),
...  'summary': None,
...  'truncated': False}
>>> util.assert_deepequal(expected, got)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "util.py", line 335, in assert_deepequal
    assert_deepequal(e_sub, g_sub, doc, stack + (key,))
  File "util.py", line 323, in assert_deepequal
    assert_deepequal(e_sub, g_sub, doc, stack + (i,))
  File "util.py", line 329, in assert_deepequal
    doc, sorted(missing), sorted(extra), expected, got, stack
AssertionError: assert_deepequal: dict keys mismatch.

  missing keys = ['nsrecord']
  extra keys = ['txtrecord']
  expected = {'dn': ipapython.dn.DN('idnsname=dnszone.test,cn=dns,dc=ipa,dc=test'), 'nsrecord': (u'ns1.dnszone.test.',), 'idnsname': [u'@']}
  got = {'dn': u'idnsname=_kerberos,idnsname=dnszone.test,cn=dns,dc=ipa,dc=test', 'txtrecord': (u'IPA.TEST',), 'idnsname': (u'_kerberos',)}
  path = ('result', 2)

The problem seems to be that {{{ assert_deepequal() }}} sorts its arguments before the recursive call to itself (https://git.fedorahosted.org/cgit/freeipa.git/tree/tests/util.py#n319). In the example above, the arguments are lists of dictionaries. Sorting a list of dictionaries does not make much sense. In fact, in Python3, a dict is considered to be an unorderable type, and attempting to sort dictionaries throws an exception. Python2 does attempt to sort dictionaries, but in the above example the first and the second list get sorted differently, which causes the keys mismatch error (see example code below).

The simplest solution for this would be simply not to call {{{ sorted()}}} if list elements are dictionaries. It would then be left to the test writer to ensure the proper order of list element in the test. In other words, the test writer must make sure the order of list elements in the test matches the order of elements as returned by the function/command that is tested.

>>> pprint.pprint(expected['result'])
[{'dn': ipapython.dn.DN('idnsname=dnszone.test,cn=dns,dc=ipa,dc=test'),
  'idnsname': [u'@'],
  'nsrecord': (u'ns1.dnszone.test.',)},
 {'dn': ipapython.dn.DN('idnsname=_kerberos,idnsname=dnszone.test,cn=dns,dc=ipa,dc=test'),
  'idnsname': [u'_kerberos'],
  'txtrecord': [u'IPA.TEST']},
 {'arecord': [u'1.2.3.4'],
  'dn': ipapython.dn.DN('idnsname=ns1,idnsname=dnszone.test,cn=dns,dc=ipa,dc=test'),
  'idnsname': [u'ns1']},
 {'arecord': [u'127.0.0.1'],
  'dn': ipapython.dn.DN('idnsname=testdnsres,idnsname=dnszone.test,cn=dns,dc=ipa,dc=test'),
  'idnsname': [u'testdnsres']}]
>>> 
>>> pprint.pprint(got['result'])
({'dn': u'idnsname=dnszone.test,cn=dns,dc=ipa,dc=test',
  'idnsname': (u'@',),
  'nsrecord': (u'ns1.dnszone.test.',)},
 {'dn': u'idnsname=_kerberos,idnsname=dnszone.test,cn=dns,dc=ipa,dc=test',
  'idnsname': (u'_kerberos',),
  'txtrecord': (u'IPA.TEST',)},
 {'arecord': (u'1.2.3.4',),
  'dn': u'idnsname=ns1,idnsname=dnszone.test,cn=dns,dc=ipa,dc=test',
  'idnsname': (u'ns1',)},
 {'arecord': (u'127.0.0.1',),
  'dn': u'idnsname=testdnsres,idnsname=dnszone.test,cn=dns,dc=ipa,dc=test',
  'idnsname': (u'testdnsres',)})
>>> 
>>> pprint.pprint(sorted(expected['result']))
[{'arecord': [u'1.2.3.4'],
  'dn': ipapython.dn.DN('idnsname=ns1,idnsname=dnszone.test,cn=dns,dc=ipa,dc=test'),
  'idnsname': [u'ns1']},
 {'arecord': [u'127.0.0.1'],
  'dn': ipapython.dn.DN('idnsname=testdnsres,idnsname=dnszone.test,cn=dns,dc=ipa,dc=test'),
  'idnsname': [u'testdnsres']},
 {'dn': ipapython.dn.DN('idnsname=dnszone.test,cn=dns,dc=ipa,dc=test'),
  'idnsname': [u'@'],
  'nsrecord': (u'ns1.dnszone.test.',)},
 {'dn': ipapython.dn.DN('idnsname=_kerberos,idnsname=dnszone.test,cn=dns,dc=ipa,dc=test'),
  'idnsname': [u'_kerberos'],
  'txtrecord': [u'IPA.TEST']}]
>>> 
>>> pprint.pprint(sorted(got['result']))
[{'arecord': (u'1.2.3.4',),
  'dn': u'idnsname=ns1,idnsname=dnszone.test,cn=dns,dc=ipa,dc=test',
  'idnsname': (u'ns1',)},
 {'arecord': (u'127.0.0.1',),
  'dn': u'idnsname=testdnsres,idnsname=dnszone.test,cn=dns,dc=ipa,dc=test',
  'idnsname': (u'testdnsres',)},
 {'dn': u'idnsname=_kerberos,idnsname=dnszone.test,cn=dns,dc=ipa,dc=test',
  'idnsname': (u'_kerberos',),
  'txtrecord': (u'IPA.TEST',)},
 {'dn': u'idnsname=dnszone.test,cn=dns,dc=ipa,dc=test',
  'idnsname': (u'@',),
  'nsrecord': (u'ns1.dnszone.test.',)}]

Metadata Update from @akrivoka:
- Issue assigned to akrivoka
- Issue set to the milestone: FreeIPA 3.2 - 2013/04-05 (GA)

7 years ago

Login to comment on this ticket.

Metadata