#5923 Inconsistent failure response in Role Plugin in python 3
Closed: fixed a year ago Opened 3 years ago by placko.

Adding non-existent privilege to role correctly results in privilege not found, but name of non-existing privilege in response is of type bytes. Other failure responses returns values as strings. Excerpt from error message of new role tests (not in master yet):

E           AssertionError: assert_deepequal: type(expected) is not type(got).
E             
E             type(expected) = <class 'str'>
E             type(got) = <class 'bytes'>
E             expected = '0fb2d4fb-ffc8-4230-9478-0b277d926cdf'
E             got = b'0fb2d4fb-ffc8-4230-9478-0b277d926cdf'
E             path = ('failed', 'member', 'privilege', 0, 0)

Relevant part of test:

  expected = {                                                                
    'completed': 0,                                                         
    'failed': {                                                             
      'member': {                                                         
        'privilege': [[str(privilege), u'privilege not found']]         
      }                                                                   
    },                                                                      
    'result': self.filter_attrs(self.privilege_keys)                        
  }                                                                           
  assert_deepequal(expected, result)

where result is result of privilege-add command.


IMO the same issue as reported in devel-list:

# ipa privilege-add-permission loc --permission='System: Show IPA Locations'

  Privilege name: loc
  Permissions: System: Modify IPA Locations
  Granting privilege to roles: loc
  Failed members:
ipa: ERROR: TypeError: sequence item 0: expected str instance, bytes found
Traceback (most recent call last):
  File "/usr/lib/python3.5/site-packages/ipalib/cli.py", line 1348, in run
    sys.exit(api.Backend.cli.run(argv))
  File "/usr/lib/python3.5/site-packages/ipalib/cli.py", line 1111, in run
    rv = cmd.output_for_cli(self.api.Backend.textui, result, *args, **options)
  File "/usr/lib/python3.5/site-packages/ipalib/frontend.py", line 1074, in
output_for_cli
    textui.print_entry(result, order, labels, flags, print_all)
  File "/usr/lib/python3.5/site-packages/ipalib/cli.py", line 391, in print_entry
    indent=indent+1
  File "/usr/lib/python3.5/site-packages/ipalib/cli.py", line 402, in print_entry
    label, value, format, indent, one_value_per_line
  File "/usr/lib/python3.5/site-packages/ipalib/cli.py", line 306, in
print_attribute
    text = ': '.join(l)
TypeError: sequence item 0: expected str instance, bytes found
ipa: ERROR: an internal error has occurred

Framework returns members as bytes

        "failed": {
            "member": {
                "permission": [
                    [
                        {
                            "__base64__": "bm9ub25v"
                        },
                        "permission not found"
                    ]
                ]
            }
        },

I suspect that this happens because in LDAP{Add,Remove}ReverseMember in this part

        for attr in options.get(self.reverse_attr) or []:
            try:
                options = {'%s' % self.member_attr: keys[-1]}
                try:
                    result = self._exc_wrapper(keys, options, self.api.Command[self.member_command])(attr, **options)
                    if result['completed'] == 1:
                        completed = completed + 1
                    else:
                        failed['member'][self.reverse_attr].append((attr, result['failed']['member'][self.member_attr][0][1]))
                except errors.NotFound as e:
                    msg = str(e)
                    (attr, msg) = msg.split(':', 1)
                    failed['member'][self.reverse_attr].append((attr, unicode(msg.strip())))

            except errors.PublicError as e:
                failed['member'][self.reverse_attr].append((attr, unicode(e)))

the variable attr is not unicode but str in py2, that is processed as bytes and send as base64 to client API. In option attributes are stored as string not unicode.

BUT when I print content of options, it contains only unicode:

{'permission': (u'permission_name',), ....}

So what is wrong? This:

                except errors.NotFound as e:
                    msg = str(e)  <------------------------- *here*
                    (attr, msg) = msg.split(':', 1) <---------------- *here*
                    failed['member'][self.reverse_attr].append((attr, unicode(msg.strip())))
  1. should be unicode as everything else in code
  2. why we parse string representation of msg, if needed exception should contain attribute which should be used
  3. why attr variable is shadowed from outer space? (Nice to debug)
  4. why we use value from exception, instead of the same value provided from options (user wants to add <permission> why we except different name to be returned by exception)

I need answers to this questions to be able create a proper fix.

Bonus: variable options should not be shadowed too, this may be again nice to debug issue in future

Fixing this both test issues and issue from comment 1 will be fixed.

Majority of that code is copy&pasted between LDAPAddReverseMember and LDAPRemoveReverseMember, would be also nice to have it just once

This is not trivial priority

This ticket is out of scope of 4.4.0 release. Moving to 4.4.1. Note that 4.4.1 needs to be triaged, therefore not everything will be implemented.

Moving to next major version. Fixing this bug is not critical in stabilization release.

mass moving python3 tickets to FreeIPA 4.6 which should be smaller release targeted mainly on python3 porting.

Metadata Update from @placko:
- Issue assigned to mbasti
- Issue set to the milestone: FreeIPA 4.6

2 years ago

Metadata Update from @mbasti:
- Issue close_status updated to: None
- Issue tagged with: py3

2 years ago

Metadata Update from @mbasti:
- Issue assigned to stlaz (was: mbasti)

2 years ago

Metadata Update from @tkrizek:
- Issue set to the milestone: FreeIPA 4.6.1 (was: FreeIPA 4.6)

2 years ago

Metadata Update from @tkrizek:
- Issue set to the milestone: FreeIPA 4.6.2 (was: FreeIPA 4.6.1)

2 years ago

Metadata Update from @tdudlak:
- Issue set to the milestone: FreeIPA 4.6.3 (was: FreeIPA 4.6.2)

2 years ago

Metadata Update from @rcritten:
- Issue set to the milestone: FreeIPA 4.6.4 (was: FreeIPA 4.6.3)

2 years ago

FreeIPA 4.6.3 has been released, moving to FreeIPA 4.6.4 milestone

I can no longer reproduce the error with FreeIPA 4.7-dev and 4.6.3. The commands no longer fail with a Python exception:

$ ipa privilege-add-permission loc --permission='System: Show IPA Locations'
ipa: ERROR: loc: privilege not found
$ ipa privilege-add loc
---------------------
Added privilege "loc"
---------------------
  Privilege name: loc
$ ipa privilege-add-permission loc --permission='System: Show IPA Locations'
  Privilege name: loc
  Failed members: 
    permission: System: Show IPA Locations: permission not found
-----------------------------
Number of permissions added 0
-----------------------------

In -vv mode, the JSON blob no longer contains bytes but proper text strings for the payload:

    "result": {
        "completed": 0,   
        "failed": {
            "member": {   
                "permission": [
                    [
                        "System",
                        "Show IPA Locations: permission not found"
                    ]
                ]
            }
        },

Metadata Update from @cheimes:
- Custom field on_review adjusted to https://github.com/freeipa/freeipa/pull/1960 (was: 1)

a year ago

master:

  • 1e5c3d7 Reproducer for issue 5923 (bytes in error response)

Metadata Update from @rcritten:
- Issue set to the milestone: FreeIPA 4.6.5 (was: FreeIPA 4.6.4)

a year ago

I'm closing the issue as fixed. The problem is no longer reproducible.

Metadata Update from @cheimes:
- Issue close_status updated to: fixed
- Issue status updated to: Closed (was: Open)

a year ago

Login to comment on this ticket.

Metadata