389-ds-base

Clone
Source Code
GIT

#50473 Issue 49239 - Add a new CI test case
Closed 4 years ago by spichugi. Opened 5 years ago by aadhikari.
aadhikari/389-ds-base tombstone-check-test  into  master

file modified
+32 
@@ -432,6 +432,38 @@ 
 

      assert "Failed to validate suffix" in result[0]
 

  

  

+ def test_check_missing_tombstones(topo_tls_ldapi):
 

+     """Check missing tombstone entries is not reported.
 

+ 

+     :id: 93067a5a-416e-4243-9418-c4dfcf42e093
 

+     :setup: Two master replication
 

+     :steps:
 

+         1. Pause replication between master and replica
 

+         2. Add and delete an entry on the master
 

+         3. Run ds-replcheck
 

+         4. Verify there are NO complaints about missing entries/tombstones
 

+     :expectedresults:
 

+         1. It should be successful
 

+         2. It should be successful
 

+         3. It should be successful
 

+         4. It should be successful
 

+     """
 

+     m1 = topo_tls_ldapi.ms["master1"]
 

+     m2 = topo_tls_ldapi.ms["master2"]
 

+ 

+     try:
 

+         topo_tls_ldapi.pause_all_replicas()
 

+         users_m1 = UserAccounts(m1, DEFAULT_SUFFIX)
 

+         user0 = users_m1.create_test_user(1000)
 

+         user0.delete()
 

+         for tool_cmd in replcheck_cmd_list(topo_tls_ldapi):
 

+             result = subprocess.check_output(tool_cmd, encoding='utf-8').lower()
 

+             assert "entries missing on replica" not in result
 

+ 

+     finally:
 

+         topo_tls_ldapi.resume_all_replicas()
 

+ 

+ 

  if __name__ == '__main__':
 

      # Run isolated
 

      # -s for DEBUG mode

file modified
+1 -1 
@@ -142,7 +142,7 @@ 
 

              continue
 

  

          # lowercase all the objectclass values (easier for tombstone checking)
 

-         oc_vals = new_entry.data['objectclass']
 

+         oc_vals = ensure_list_str(new_entry.data['objectclass'])
 

          new_oc_vals = []
 

          for val in oc_vals:
 

              new_oc_vals.append(val.lower())

Bug Description: ds-replcheck unreliable, showing false positives, showing missing tombstone entries
in the report.

Fix Description: Added a test case to check missing tombstone entries is not reported

https://pagure.io/389-ds-base/issue/49239

Review by: ???

rebased onto f9d03ea27fdad09839fbf04ec91e1d85fd2c6463
5 years ago

This test fails on master:

>               assert "entries missing on replica" not in result                                                                                                                                                                                                                
E               AssertionError: assert 'entries missing on replica' not in '============... replica\n\n'                                                                                                                                                                         
E                 'entries missing on replica' is contained here:                                                                                                                                                                                                                
E                   ================================================================================                                                                                                                                                                             
E                            replication synchronization report  (tue jul  2 18:51:13 2019)                                                                                                                                                                                      
E                   ================================================================================                                                                                                                                                                             
E                                                                                                                                                                                                                                                                                
E                                                                                                                                                                                                                                                                                
E                   database ruv's                                                                                                                                                                                                                                               
E                   =====================================================                                                                                                                                                                                                        
E                                                                                                                                                                                                                                                                                
E                   master ruv:                                                                                                                                                                                                                                                  
E                     {replica 1 ldap://server-f30.example.com:39001} 5d1ba76a000100010000 5d1ba790000100010000                                                                                                                                                                  
E                     {replica 2 ldap://server-f30.example.com:39002} 5d1ba771000100020000 5d1ba78f000000020000                                                                                                                                                                  
E                     {replicageneration} 5d1ba76a000000010000                                                                                                                                                                                                                   
E                                                                                                                                                                                                                                                                                
E                   replica ruv:                                                                                                                                                                                                                                                 
E                     {replica 1 ldap://server-f30.example.com:39001} 5d1ba76a000100010000 5d1ba78e000000010000                                                                                                                                                                  
E                     {replica 2 ldap://server-f30.example.com:39002} 5d1ba771000100020000 5d1ba78f000000020000                                                                                                                                                                  
E                     {replicageneration} 5d1ba76a000000010000                                                                                                                                                                                                                   
E                                                                                                                                                                                                                                                                                
E                   replication state: replica is behind master by: 2 seconds                                                                                                                                                                                                    
E                                                                                                                                                                                                                                                                                
E                                                                                                                                                                                                                                                                                
E                   entry counts                                                                                                                                                                                                                                                 
E                   =====================================================                                                                                                                                                                                                        
E                                                                                                                                                                                                                                                                                
E                   master:  15                                                                                                                                                                                                                                                  
E                   replica: 14                                                                                                                                                                                                                                                  
E                                                                                                                                                                                                                                                                                
E                                                                                                                                                                                                                                                                                
E                   tombstones                                                                                                                                                                                                                                                   
E                   =====================================================                                                                                                    
E                                                                                                                                                                            
E                   master:  1                                                                                                                                               
E                   replica: 0                                                                                                                                               
E                                                                                                                                                                            
E                   missing entries                                                                                                                                          
E                   =====================================================                                                                                                    
E                                                                                                                                                                            
E                     entries missing on replica:                                                                                                                            
E                      - nsuniqueid=4e3eb401-9cfa11e9-838df36e-d9df3ae9,uid=test_user_1000,ou=people,dc=example,dc=com  (created on master at: tue jul  2 18:50:56 2019)     
E                                                                                                                                                                            
E                   result                                                                                                                                                   
E                   =====================================================                                                                                                    
E                                                                                                                                                                            
E                   there are replication differences between master and replica                                                                                             

dirsrvtests/tests/suites/ds_tools/replcheck_test.py:432: AssertionError

@mreynolds, I thought it was fixed. Any idea why it fails?

What test condition is failing? There are only 409 lines in the test script, so I don't know where line 432 is coming from. Perhaps this test was run from a different/older branch?

It appears the problem is related to python3 and how it allows for checking arrays.

This works correctly in python2:

if 'nstombstone' in entry.data['objectclass']:

But in python3 it must be:

if b'nstombstone' in entry.data['objectclass']:

or

if 'nstombstone' in map(bytes.decode, entry.data['objectclass']):

Working on fix, but this only impacts 389-ds-base-1.4.x (RHEL 8)

@aadhikari - if you want you can add this code to your PR to fix the problem. Or I can file a separate ticket...

diff --git a/ldap/admin/src/scripts/ds-replcheck b/ldap/admin/src/scripts/ds-replcheck
index 4abb417af..30bcfd65d 100755
--- a/ldap/admin/src/scripts/ds-replcheck
+++ b/ldap/admin/src/scripts/ds-replcheck
@@ -142,7 +142,7 @@ def convert_entries(entries):
             continue

         # lowercase all the objectclass values (easier for tombstone checking)
-        oc_vals = new_entry.data['objectclass']
+        oc_vals = ensure_list_str(new_entry.data['objectclass'])
         new_oc_vals = []
         for val in oc_vals:
             new_oc_vals.append(val.lower())

@mreynolds No problem at all, I will put it with this PR, Thanks for the help!

rebased onto 2a17814275782944d091992b0e5b973d81b339f2
5 years ago

rebased onto c2650f0
5 years ago

@mreynolds It's done, @vashirov, please review.

LGTM, thanks!

Pull-Request has been merged by vashirov
5 years ago

389-ds-base is moving from Pagure to Github. This means that new issues and pull requests
will be accepted only in 389-ds-base's github repository.

This pull request has been cloned to Github as issue and is available here:
- https://github.com/389ds/389-ds-base/issues/3530

If you want to continue to work on the PR, please navigate to the github issue,
download the patch from the attachments and file a new pull request.

Thank you for understanding. We apologize for all inconvenience.

Pull-Request has been closed by spichugi
4 years ago
Metadata
None
No Tags
Changes Summary 2
+32 -0
file changed
dirsrvtests/tests/suites/ds_tools/replcheck_test.py
+1 -1
file changed
ldap/admin/src/scripts/ds-replcheck
Powered by Pagure 5.14.1
Documentation • File an Issue • About • SSH Hostkey/Fingerprint
© Red Hat, Inc. and others.