Running ipa-run-tests-3 test_integration/test_simple_replication.py raises "bytes-like object is required, not 'str'" as described below.
ipa-run-tests-3 test_integration/test_simple_replication.py
"bytes-like object is required, not 'str'"
[...] [ipatests.pytest_plugins.integration.host.Host.vm-058-021] Connecting to LDAP at vm-058-021.abc.idm.lab.eng.brq.redhat.com [ipatests.pytest_plugins.integration.host.Host.vm-058-021] LDAP bind as cn=Directory Manager FAILED >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> traceback >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> self = <ipatests.test_integration.test_simple_replication.TestSimpleReplication object at 0x7fe62582ac88> def test_user_replication_to_replica(self): """Test user replication master -> replica""" > self.check_replication(self.master, self.replicas[0], 'testuser1') test_integration/test_simple_replication.py:65: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ test_integration/test_simple_replication.py:45: in check_replication tasks.wait_for_replication(source_ldap) pytest_plugins/integration/tasks.py:1071: in wait_for_replication logger.debug('Replication agreements: \n%s', _entries_to_ldif(entries)) pytest_plugins/integration/tasks.py:1048: in _entries_to_ldif writer.unparse(str(entry.dn), dict(entry)) /usr/lib64/python3.6/site-packages/ldif.py:210: in unparse self._unparseAttrTypeandValue('dn', dn) /usr/lib64/python3.6/site-packages/ldif.py:158: in _unparseAttrTypeandValue self._unfold_lines(': '.join([attr_type, attr_value.decode('ascii')])) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <ldif.LDIFWriter object at 0x7fe6243c0550> line = 'dn: cn=meTovm-058-025.abc.idm.lab.eng.brq.redhat.com,cn=replica,cn=dc\\=ipa\\,dc\\=test,cn=mapping tree,cn=config' def _unfold_lines(self,line): """ Write string line as one or more folded lines """ # Check maximum line length line_len = len(line) if line_len<=self._cols: self._output_file.write(line) self._output_file.write(self._last_line_sep) else: # Fold line pos = self._cols > self._output_file.write(line[0:min(line_len,self._cols)]) E TypeError: a bytes-like object is required, not 'str' /usr/lib64/python3.6/site-packages/ldif.py:126: TypeError >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> entering PDB >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > /usr/lib64/python3.6/site-packages/ldif.py(126)_unfold_lines() -> self._output_file.write(line[0:min(line_len,self._cols)])
The error happens in pytest_plugins/integration/tasks.py, here:
pytest_plugins/integration/tasks.py
from ldif import LDIFWriter def _entries_to_ldif(entries): io = StringIO() writer = LDIFWriter(io) for entry in entries: writer.unparse(str(entry.dn), dict(entry)) # here return io.getvalue()
Doing some quick debugging I discovered: using ldif3 and changing StringIO to BytesIO solves the problem. So, maybe there is a bug in python-ldap, since pyldap call it here
StringIO
BytesIO
from ldif3 import LDIFWriter def _entries_to_ldif(entries): io = BytesIO() writer = LDIFWriter(io) for entry in entries: writer.unparse(str(entry.dn), dict(entry)) return io.getvalue()
More debug info:
-> writer.unparse(str(entry.dn), dict(entry)) (Pdb) writer.unparse(str(entry.dn), dict(entry)) *** TypeError: a bytes-like object is required, not 'str' (Pdb) writer.unparse(b'blah', {b'name':b'blah'}) *** AttributeError: 'bytes' object has no attribute 'encode' (Pdb) writer.unparse(b'blah'.decode(), {b'name':b'blah'}) *** TypeError: expected string or bytes-like object (Pdb) writer.unparse(b'blah'.decode(), dict(entry)) *** TypeError: cannot use a bytes pattern on a string-like object (Pdb) writer.unparse(str(entry.dn).encode(), dict(entry)) *** AttributeError: 'bytes' object has no attribute 'encode'
Marking as test-failure as it will be if used.
Metadata Update from @pvoborni: - Issue priority set to: normal - Issue set to the milestone: FreeIPA 4.6.2 - Issue tagged with: test-failure
Metadata Update from @stlaz: - Issue assigned to stlaz
This is being fixed as a part of https://github.com/freeipa/freeipa/pull/1112
Metadata Update from @stlaz: - Custom field on_review adjusted to https://github.com/freeipa/freeipa/pull/1112
Please note that the statement:
is false and the bug is on our side since we're presenting wrong type of data to the LDIFWriter, which expects all attribute values to be bytes.
Metadata Update from @tdudlak: - Issue set to the milestone: FreeIPA 4.6.3 (was: FreeIPA 4.6.2)
Metadata Update from @rcritten: - Issue set to the milestone: FreeIPA 4.6.4 (was: FreeIPA 4.6.3)
FreeIPA 4.6.3 has been released, moving to FreeIPA 4.6.4 milestone
master:
a93592a PRCI: use a new template for py3 testing d39456a ipatests: use python3 if built with python3 71a8026 py3: pass raw entries to LDIFWriter
ipa-4-6:
c166792 PRCI: use a new template for py3 testing 9baa3f6 ipatests: use python3 if built with python3 5a2b428 py3: pass raw entries to LDIFWriter
Fixed in 4.6.2. Moving milestone and closing.
Metadata Update from @rcritten: - Issue close_status updated to: fixed - Issue set to the milestone: FreeIPA 4.6.2 (was: FreeIPA 4.6.4) - Issue status updated to: Closed (was: Open)
Python makes a clear distinction between bytes and strings . Bytes objects contain raw data — a sequence of octets — whereas strings are Unicode sequences . Conversion between these two types is explicit: you encode a string to get bytes, specifying an encoding (which defaults to UTF-8); and you decode bytes to get a string. Clients of these functions should be aware that such conversions may fail, and should consider how failures are handled.
We can convert bytes to string using bytes class decode() instance method, So you need to decode the bytes object to produce a string. In Python 3 , the default encoding is "utf-8" , so you can use directly:
b"python byte to string".decode("utf-8")
Login to comment on this ticket.