389-ds-base

Clone
Source Code
GIT

#50364 Ticket 50363 - ds-replcheck incorrectly reports error for out of order multi-valued attributes
Closed 3 years ago by spichugi. Opened 4 years ago by mreynolds.
mreynolds/389-ds-base ticket50363  into  master

file modified
+9 
@@ -343,6 +343,8 @@ 
 

      m2 = topo_tls_ldapi.ms["master2"]
 

      attr_m1 = "m1_inconsistency"
 

      attr_m2 = "m2_inconsistency"
 

+     attr_first = "first ordered valued"
 

+     attr_second = "second ordered valued"
 

      attr_m1_only = "123123123"
 

  

      try:
 
@@ -355,6 +357,9 @@ 
 

          user_m1.set("description", attr_m1)
 

          user_m2.set("description", attr_m2)
 

          user_m1.set("telephonenumber", attr_m1_only)
 

+         # Add the same multi-valued attrs, but out of order
 

+         user_m1.set("cn", [attr_first, attr_second])
 

+         user_m2.set("cn", [attr_second, attr_first])
 

          time.sleep(2)
 

  

          for tool_cmd in replcheck_cmd_list(topo_tls_ldapi):
 
@@ -362,12 +367,16 @@ 
 

              assert attr_m1 in result
 

              assert attr_m2 in result
 

              assert attr_m1_only in result
 

+             assert attr_first not in result
 

+             assert attr_second not in result
 

              # Ignore some attributes and check the output
 

              tool_cmd.extend(['-i', '{},{}'.format('description', 'telephonenumber')])
 

              result = subprocess.check_output(tool_cmd, encoding='utf-8').lower()
 

              assert attr_m1 not in result
 

              assert attr_m2 not in result
 

              assert attr_m1_only not in result
 

+             assert attr_first not in result
 

+             assert attr_second not in result
 

  

      finally:
 

          topo_tls_ldapi.resume_all_replicas()

file modified
+6 -1 
@@ -125,7 +125,7 @@ 
 

  

      for entry in entries:
 

          new_entry = Entry(entry)
 

-         new_entry.data = {k.lower(): v for k, v in list(new_entry.data.items())}
 

+         new_entry.data = {k.lower(): sorted(v) for k, v in list(new_entry.data.items())}
 

  

          # Decode nscpentrywsi bytes values for future use
 

          nscpentrywsi_list = []
 
@@ -448,6 +448,11 @@ 
 

      # Keep track of entry index - we use this later when searching the LDIF again
 

      result['idx'] = count
 

  

+     # Sort all the multi-valued attributes
 

+     for k, v in data.items():
 

+         v.sort()
 

+         data[k] = v
 

+ 

      result['glue'] = None
 

      if found_conflict and found_subentry and found_tombstone is False:
 

          result['entry'] = None

Bug Description:

If for some reason an entry's multi-valued attribute values are in different orders on different replicas the tool reports this as an inconsistency when it is not.

Fix Description:

For both offline & online processing sort each entry's multi-valued attribute values.

https://pagure.io/389-ds-base/issue/50363

I think this fix is okay, but curious if there is possibly a way to sort these in the lib389 tools in mapped_object or similar so we avoid this problem? Ack if you just want merge though.

We could do here something like ...{k.lower(): sorted(v) for... instead of the block below, I guess. Something similar in the next added block as well, probably.

We could do here something like ...{k.lower(): sorted(v) for... instead of the block below, I guess. Something similar in the next added block as well, probably.

Yeah I tried: {k.lower(): v.sort() for...} but it didn't work. I don't fully understand these type of "for loops" - need to re-read that stuff :-) I'll give this a shot, thanks!

@firstyear

I think this fix is okay, but curious if there is possibly a way to sort these in the lib389 tools in mapped_object or similar so we avoid this problem?

Well this is outside of mapped objects in this CLI tool (ds-replcheck). It calling raw search_s and getting an "Entry". And in the other case it's built from an LDIF file (no search - it's all manually built), but I might be able to use the LDIF module for that. I'll open a new ticket to refactor the tool for mapped objects, and investigate using the LDIF module.
Edited 4 years ago by mreynolds

rebased onto 6c805f344e511fd919c84043481ef21cd24f47fe
4 years ago

rebased onto 974c802
4 years ago

Pull-Request has been merged by mreynolds
4 years ago

Cool, no problems then :) just wanted to be sure this was checked as you are more familar with this code than I am.

389-ds-base is moving from Pagure to Github. This means that new issues and pull requests
will be accepted only in 389-ds-base's github repository.

This pull request has been cloned to Github as issue and is available here:
- https://github.com/389ds/389-ds-base/issues/3423

If you want to continue to work on the PR, please navigate to the github issue,
download the patch from the attachments and file a new pull request.

Thank you for understanding. We apologize for all inconvenience.

Pull-Request has been closed by spichugi
3 years ago
Metadata
None
No Tags
Changes Summary 2
+9 -0
file changed
dirsrvtests/tests/suites/ds_tools/replcheck_test.py
+6 -1
file changed
ldap/admin/src/scripts/ds-replcheck
Powered by Pagure 5.13.3
Documentation • File an Issue • About • SSH Hostkey/Fingerprint
© Red Hat, Inc. and others.