I'm attaching LDIF for reproducing the problem.
URL /v1/ldap/fqdn=vm-205.abc.idm.lab.eng.brq.redhat.com,cn=computers,cn=accounts,dc=ipa,dc=example/sub/(objectClass=*)/krbExtraData always return a traceback:
/v1/ldap/fqdn=vm-205.abc.idm.lab.eng.brq.redhat.com,cn=computers,cn=accounts,dc=ipa,dc=example/sub/(objectClass=*)/krbExtraData
UnicodeDecodeError UnicodeDecodeError: 'utf8' codec can't decode byte 0xe8 in position 4: invalid continuation byte Traceback (most recent call last) File "/usr/lib/python2.7/site-packages/Flask-0.10.1-py2.7.egg/flask/app.py", line 1836, in __call__ return self.wsgi_app(environ, start_response) File "/usr/lib/python2.7/site-packages/Flask-0.10.1-py2.7.egg/flask/app.py", line 1820, in wsgi_app response = self.make_response(self.handle_exception(e)) File "/usr/lib/python2.7/site-packages/Flask-0.10.1-py2.7.egg/flask/app.py", line 1403, in handle_exception reraise(exc_type, exc_value, tb) File "/usr/lib/python2.7/site-packages/Flask-0.10.1-py2.7.egg/flask/app.py", line 1817, in wsgi_app response = self.full_dispatch_request() File "/usr/lib/python2.7/site-packages/Flask-0.10.1-py2.7.egg/flask/app.py", line 1477, in full_dispatch_request rv = self.handle_user_exception(e) File "/usr/lib/python2.7/site-packages/Flask-0.10.1-py2.7.egg/flask/app.py", line 1381, in handle_user_exception reraise(exc_type, exc_value, tb) File "/usr/lib/python2.7/site-packages/Flask-0.10.1-py2.7.egg/flask/app.py", line 1475, in full_dispatch_request rv = self.dispatch_request() File "/usr/lib/python2.7/site-packages/Flask-0.10.1-py2.7.egg/flask/app.py", line 1461, in dispatch_request return self.view_functions[rule.endpoint](**req.view_args) File "/root/rest389/rest389/Authentication.py", line 164, in decorator return _connection(f, *args, **kwargs) File "/root/rest389/rest389/Authentication.py", line 145, in _connection retval = f(dir_srv_conn, *args, **kwargs) File "/root/rest389/rest389/LDAPOps.py", line 371, in search_SFA return jsonReturnEntries(entries, request.url) File "/root/rest389/rest389/LDAPOps.py", line 62, in jsonReturnEntries return jsonResponse(results, 200, link) File "/root/rest389/rest389/Helpers.py", line 18, in jsonResponse resp = jsonify(resp_msg) File "/usr/lib/python2.7/site-packages/Flask-0.10.1-py2.7.egg/flask/json.py", line 238, in jsonify indent=indent), File "/usr/lib/python2.7/site-packages/Flask-0.10.1-py2.7.egg/flask/json.py", line 126, in dumps rv = _json.dumps(obj, **kwargs) File "/usr/lib64/python2.7/site-packages/simplejson/__init__.py", line 382, in dumps **kw).encode(obj) File "/usr/lib64/python2.7/site-packages/simplejson/encoder.py", line 271, in encode chunks = list(chunks) File "/usr/lib64/python2.7/site-packages/simplejson/encoder.py", line 632, in _iterencode for chunk in _iterencode_dict(o, _current_indent_level): File "/usr/lib64/python2.7/site-packages/simplejson/encoder.py", line 591, in _iterencode_dict for chunk in chunks: File "/usr/lib64/python2.7/site-packages/simplejson/encoder.py", line 481, in _iterencode_list for chunk in chunks: File "/usr/lib64/python2.7/site-packages/simplejson/encoder.py", line 591, in _iterencode_dict for chunk in chunks: File "/usr/lib64/python2.7/site-packages/simplejson/encoder.py", line 591, in _iterencode_dict for chunk in chunks: File "/usr/lib64/python2.7/site-packages/simplejson/encoder.py", line 450, in _iterencode_list yield buf + _encoder(value) UnicodeDecodeError: 'utf8' codec can't decode byte 0xe8 in position 4: invalid continuation byte
Tested with rest389 2cea971040396e73802aae9854b4dfa9523ee2b0.
attachment computers.ldif
I don't like this. I think that this patch should not be in rest389. Because of where the code is, we'll have to have the decodeEntry part around all over the place.
I would rather have this in lib389/_entry.py so that on any operation the decode is implicit and transparent. It will also be useful to have this in lib389 anyway.
If we commit to this way, we'll end up with messy split up code for managing entries between rest389 and lib389.
Rest389 should do nothing to entries, or the data in them, it should never implement an operation, it should be the most minimal wrapper possible to enable a rest connection into lib389. It shouldn't even be data massaging for formatting IMO.
Anyway, sorry, but it's nack from me :(
Replying to [comment:3 firstyear]:
I don't like this. I think that this patch should not be in rest389. Because of where the code is, we'll have to have the decodeEntry part around all over the place. I would rather have this in lib389/_entry.py so that on any operation the decode is implicit and transparent. It will also be useful to have this in lib389 anyway.
I completely agree - I'll move this function to lib389. Note - it's only JSON dump that has an issue with the binary values
the function converts a binary to a base64 string, so I would say it is encoding, not decoding
it looks like only the first value of an attr is encoded, how about multivalued attributes ?
Replying to [comment:5 lkrispen]:
I'll revise the function name to something more accurate.
It currently handles multivalued attributes just fine.
Replying to [comment:6 mreynolds]:
Replying to [comment:5 lkrispen]: it looks like only the first value of an attr is encoded, how about multivalued attributes ? It currently handles multivalued attributes just fine. but attr.getValue() just returns one value, shouldn't it be
It currently handles multivalued attributes just fine. but attr.getValue() just returns one value, shouldn't it be
for v in attr.getValues()
Replying to [comment:7 lkrispen]:
Replying to [comment:6 mreynolds]: Replying to [comment:5 lkrispen]: it looks like only the first value of an attr is encoded, how about multivalued attributes ? It currently handles multivalued attributes just fine. but attr.getValue() just returns one value, shouldn't it be for v in attr.getValues()
Yes, that seems correct, but the results show all the attribute values being converted with the existing fix:
ipaSshPubKey:: ZWNkc2Etc2hhMi1uaXN0cDI1NiBBQUFBRTJWalpITmhMWE5vWVRJdGJtbHpkSEF5TlRZQUFBQUlibWx6ZEhBeU5UWUFBQUJCQkJKNExZdzFpWnNCN3hJaWFKaERKek1FaGJ4bHEzNWVVVXpaVWdKcHRCaTc2RjVjbUZHUHY1MTM5MHgzMG1OeWJaR0F4UkphdXZUa1ZrclJrdElSS2prPQ== ipaSshPubKey:: c3NoLWVkMjU1MTkgQUFBQUMzTnphQzFsWkRJMU5URTVBQUFBSUlKY0I0M1BPWC93QnNCME1zWEpabVZiVnVZenJJQVM1R09iYWtBWjE5S0Y= ipaSshPubKey:: c3NoLXJzYSBBQUFBQjNOemFDMXljMkVBQUFBREFRQUJBQUFCQVFDOURkMGVNZkdXc1Rvb0pKYzZEMGVWWVVqM1NoeEEzRk93Y3VNUys0TkNmdC8ra2FiUWJtRWxpRXk1NTU3enpSZzVxOVN3RmxKdElGcEFuaTJMMnduYTc5L0NwKzVVRnhUTHV6c0toQlNtVWVwdEtvVEc3YkovclYxR09rcyt1V2RscGZqbjNlZFNieWRWbThyTWlyTFpKMUYzWUhjZHRKU0l1NlNFS2R2eGtETTFYR3YrWkM1bjVMd056Tlk5RFV0MG93bWt0WUozbUM5anFsa2VGaEg1OFVROC9UeGtlMzdwNnBmQ2dzMi9ZWFh1RHBsOC9MbkJTaHVwekZ0WDRBdDYyN1h5NStsNEdlSHVaa0o4MGRkYkpmTU9QV0d4aThhSHRNNHdLOVN5SFB1enR6UUdBV0tsOHY1TU0zWExGbXBaa05DQ1lHT0hSVEJmdkNCWVdjUHo= krbExtraData:: AAJFO+hVcm9vdC9hZG1pbkBJUEEuRVhBTVBMRQA=
To:
"ipasshpubkey": [ "ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBJ4LYw1iZsB7xIiaJhDJzMEhbxlq35eUUzZUgJptBi76F5cmFGPv51390x30mNybZGAxRJauvTkVkrRktIRKjk=", "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIIJcB43POX/wBsB0MsXJZmVbVuYzrIAS5GObakAZ19KF", "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9Dd0eMfGWsTooJJc6D0eVYUj3ShxA3FOwcuMS+4NCft/+kabQbmEliEy5557zzRg5q9SwFlJtIFpAni2L2wna79/Cp+5UFxTLuzsKhBSmUeptKoTG7bJ/rV1GOks+uWdlpfjn3edSbydVm8rMirLZJ1F3YHcdtJSIu6SEKdvxkDM1XGv+ZC5n5LwNzNY9DUt0owmktYJ3mC9jqlkeFhH58UQ8/Txke37p6pfCgs2/YXXuDpl8/LnBShupzFtX4At627Xy5+l4GeHuZkJ80ddbJfMOPWGxi8aHtM4wK9SyHPuztzQGAWKl8v5MM3XLFmpZkNCCYGOHRTBfvCBYWcPz" ],
Perhaps this is already being converted by python-ldap(its already base64). I will do more thorough testing later today.
Moving ticket to lib389
attachment 0001-Ticket-48660-Add-function-to-convert-binary-values-i.patch
Thanks for this! Looks good!
To ssh://git.fedorahosted.org/git/389/lib389.git 853d950..3f56c8d master -> master commit 3f56c8d4851ed0d99f2a0b17e51eb249e4597a55 Author: Mark Reynolds mreynolds@redhat.com Date: Tue Apr 5 16:03:49 2016 -0400
Metadata Update from @mreynolds: - Issue assigned to mreynolds - Issue set to the milestone: lib389 1.0.2
389-ds-base is moving from Pagure to Github. This means that new issues and pull requests will be accepted only in 389-ds-base's github repository.
This issue has been cloned to Github and is available here: - https://github.com/389ds/389-ds-base/issues/1789
If you want to receive further updates on the issue, please navigate to the github issue and click on subscribe button.
subscribe
Thank you for understanding. We apologize for all inconvenience.
Metadata Update from @spichugi: - Issue close_status updated to: wontfix (was: Fixed)
Log in to comment on this ticket.