389-ds-base

Clone
Source Code
GIT

#50573 Issue 50572 - After running cl-dump dbdir/cldb/*ldif.done are not deleted
Closed 3 years ago by spichugi. Opened 4 years ago by spichugi.
spichugi/389-ds-base cl-dump-fix  into  master

file modified
+15 -8 
@@ -5,7 +5,7 @@ 
 

  # All rights reserved.
 

  #
 

  # License: GPL (version 3 or any later version).
 

- # See LICENSE for details. 

+ # See LICENSE for details.
 

  # END COPYRIGHT BLOCK
 

  ###################################################################################
 

  #
 
@@ -13,7 +13,7 @@ 
 

  #
 

  # SYNOPSIS:
 

  #  cl-dump.pl [-h host] [-p port] [-D bind-dn] -w bind-password | -P bind-cert
 

- #       [-r replica-roots] [-o output-file] [-c] [-v]
 

+ #       [-r replica-roots] [-o output-file] [-c] [-l] [-v]
 

  #
 

  #  cl-dump.pl -i changelog-ldif-file-with-base64encoding [-o output-file] [-c]
 

  #
 
@@ -22,7 +22,7 @@ 
 

  #
 

  # OPTIONS:
 

  #    -c Dump and interpret CSN only. This option can be used with or
 

- #       without -i option. 

+ #       without -i option.
 

  #
 

  #    -D bind-dn
 

  #       Directory server's bind DN. Default to "cn=Directory Manager" if
 
@@ -32,6 +32,8 @@ 
 

  #       Directory server's host. Default to the server where the script
 

  #       is running.
 

  #
 

+ #    -l Preserve generated ldif.done files from changelogdir
 

+ #
 

  #    -i changelog-ldif-file-with-base64encoding
 

  #       If you already have a ldif-like changelog, but the changes
 

  #       in that file are encoded, you may use this option to
 
@@ -68,7 +70,7 @@ 
 

  # all of this nonsense can be omitted if the mozldapsdk and perldap are
 

  # installed in the operating system locations (e.g. /usr/lib /usr/lib/perl5)
 

  

- $usage="Usage: $0 [-h host] [-p port] [-D bind-dn] [-w bind-password | -P bind-cert] [-r replica-roots] [-o output-file] [-c] [-v]\n\n       $0 -i changelog-ldif-file-with-base64encoding [-o output-file] [-c]\n";
 

+ $usage="Usage: $0 [-h host] [-p port] [-D bind-dn] [-w bind-password | -P bind-cert] [-r replica-roots] [-o output-file] [-c] [-l] [-v]\n\n       $0 -i changelog-ldif-file-with-base64encoding [-o output-file] [-c]\n";
 

  

  use Getopt::Std;			# Parse command line arguments
 

  use Mozilla::LDAP::Conn;		# LDAP module for Perl
 
@@ -86,7 +88,7 @@ 
 

  	$| = 1;
 

  

  	# Check for legal options
 

- 	if (!getopts('h:p:D:w:P:r:o:cvi:')) {
 

+ 	if (!getopts('h:p:D:w:P:r:o:clvi:')) {
 

  		print $usage;
 

  		exit -1;
 

  	}
 
@@ -123,7 +125,7 @@ 
 

  	if ($opt_o && ! open (OUTPUT, ">$opt_o")) {
 

  		print "Can't create output file $opt_o\n";
 

  		$rc = -1;
 

- 	} 

+ 	}
 

  	# Open STDOUT if option -o is missing
 

  	open (OUTPUT, ">-") if !$opt_o;
 

  
@@ -194,10 +196,15 @@ 
 

  			else {
 

  				&cl_decode ($_);
 

  			}
 

- 			# Test op -M doesn't work well so we use rename
 

+ 			# Test op -M doesn't work well so we use rename/remove
 

  			# here to avoid reading the same ldif file more
 

  			# than once.
 

- 			rename ($ldif, "$ldif.done");
 

+ 			if ($opt_l) {
 

+ 				rename ($ldif, "$ldif.done");
 

+ 			} else {
 

+ 				# Remove the file - default behaviou when '-l' is not specified
 

+ 				unlink ($ldif)
 

+ 			}
 

  		}
 

  		&print_header ($replica, "Not Found") if !$gotldif;
 

  	}

file modified
+7 -4 
@@ -20,7 +20,7 @@ 
 

  .SH SYNOPSIS
 

  .B cl\-dump
 

  [\fI\-h host\fR] [\fI\-p port\fR] [\fI\-D bind\(hydn\fR] \-w bind\(hypassword | \-P bind\(hycert
 

-        [\fI\-r replica\(hyroots\fR] [\fI\-o output\(hyfile\fR] [\fI\-c\fR] [\fI\-v\fR]
 

+        [\fI\-r replica\(hyroots\fR] [\fI\-o output\(hyfile\fR] [\fI\-c\fR] [\fI\-l\fR] [\fI\-v\fR]
 

  

  .PP
 

  .B cl\-dump
 
@@ -30,12 +30,12 @@ 
 

  Dump and decode Directory Server replication change log
 

  .PP
 

  .\" TeX users may be more comfortable with the \fB<whatever>\fP and
 

- .\" \fI<whatever>\fP escape sequences to invode bold face and italics, 

+ .\" \fI<whatever>\fP escape sequences to invode bold face and italics,
 

  .\" respectively.
 

  .SH OPTIONS
 

  A summary of options is included below.
 

  .TP
 

- .B \-c 

+ .B \-c
 

  Dump and interpret CSN only. This option can be used with or
 

  without \-i option.
 

  .TP
 
@@ -47,6 +47,9 @@ 
 

  Directory server's host. Default to the server where the script
 

  is running.
 

  .TP
 

+ .B \-l
 

+ Preserve generated ldif.done files from changelogdir
 

+ .TP
 

  .B \-i changelog\(hyldif\(hyfile\(hywith\(hybase64encoding
 

  If you already have a ldif-like changelog, but the changes
 

  in that file are encoded, you may use this option to
 
@@ -66,7 +69,7 @@ 
 

  roots may be separated by comma. All the replica roots would be
 

  dumped if the option is omitted.
 

  .TP
 

- .B \-v 

+ .B \-v
 

  Print the version of this script.
 

  .TP
 

  .B \-w bind\(hypassword

file modified
+7 -4 
@@ -20,7 +20,7 @@ 
 

  .SH SYNOPSIS
 

  .B cl\-dump.pl
 

  [\fI\-h host\fR] [\fI\-p port\fR] [\fI\-D bind\(hydn\fR] \-w bind\(hypassword | \-P bind\(hycert
 

-        [\fI\-r replica\(hyroots\fR] [\fI\-o output\(hyfile\fR] [\fI\-c\fR] [\fI\-v\fR]
 

+        [\fI\-r replica\(hyroots\fR] [\fI\-o output\(hyfile\fR] [\fI\-c\fR] [\fI\-l\fR] [\fI\-v\fR]
 

  

  .PP
 

  .B cl\-dump.pl
 
@@ -30,12 +30,12 @@ 
 

  Dump and decode Directory Server replication change log
 

  .PP
 

  .\" TeX users may be more comfortable with the \fB<whatever>\fP and
 

- .\" \fI<whatever>\fP escape sequences to invode bold face and italics, 

+ .\" \fI<whatever>\fP escape sequences to invode bold face and italics,
 

  .\" respectively.
 

  .SH OPTIONS
 

  A summary of options is included below.
 

  .TP
 

- .B \-c 

+ .B \-c
 

  Dump and interpret CSN only. This option can be used with or
 

  without \-i option.
 

  .TP
 
@@ -47,6 +47,9 @@ 
 

  Directory server's host. Default to the server where the script
 

  is running.
 

  .TP
 

+ .B \-l
 

+ Preserve generated ldif.done files from changelogdir
 

+ .TP
 

  .B \-i changelog\(hyldif\(hyfile\(hywith\(hybase64encoding
 

  If you already have a ldif-like changelog, but the changes
 

  in that file are encoded, you may use this option to
 
@@ -66,7 +69,7 @@ 
 

  roots may be separated by comma. All the replica roots would be
 

  dumped if the option is omitted.
 

  .TP
 

- .B \-v 

+ .B \-v
 

  Print the version of this script.
 

  .TP
 

  .B \-w bind\(hypassword

file modified
+5 -1 
@@ -894,7 +894,9 @@ 
 

          log.addHandler(fh)
 

      replicas = Replicas(inst)
 

      if not args.changelog_ldif:
 

-         replicas.process_and_dump_changelog(replica_roots=args.replica_roots, csn_only=args.csn_only)
 

+         replicas.process_and_dump_changelog(replica_roots=args.replica_roots,
 

+                                             csn_only=args.csn_only,
 

+                                             preserve_ldif_done=args.preserve_ldif_done)
 

      else:
 

          try:
 

              assert os.path.exists(args.changelog_ldif)
 
@@ -997,6 +999,8 @@ 
 

      repl_set_cl.set_defaults(func=dump_cl)
 

      repl_set_cl.add_argument('-c', '--csn-only', action='store_true',
 

                               help="Dump and interpret CSN only. This option can be used with or without -i option.")
 

+     repl_set_cl.add_argument('-l', '--preserve-ldif-done', action='store_true',
 

+                              help="Preserve generated ldif.done files from changelogdir.")
 

      repl_set_cl.add_argument('-i', '--changelog-ldif',
 

                               help="If you already have a ldif-like changelog, but the changes in that file are encoded,"
 

                                    " you may use this option to decode that ldif-like changelog. It should be base64 encoded.")

file modified
+7 -2 
@@ -1496,7 +1496,7 @@ 
 

              replica._populate_suffix()
 

          return replica
 

  

-     def process_and_dump_changelog(self, replica_roots=[], csn_only=False):
 

+     def process_and_dump_changelog(self, replica_roots=[], csn_only=False, preserve_ldif_done=False):
 

          """Dump and decode Directory Server replication change log
 

  

          :param replica_roots: Replica suffixes that need to be processed
 
@@ -1541,7 +1541,12 @@ 
 

                      cl_ldif.grep_csn()
 

                  else:
 

                      cl_ldif.decode()
 

-                 os.rename(file_path, f'{file_path}.done')
 

+ 

+                 if preserve_ldif_done:
 

+                     os.rename(file_path, f'{file_path}.done')
 

+                 else:
 

+                     os.remove(file_path)
 

+ 

              if not got_ldif:
 

                  self._log.info("LDIF file: Not found")

Description: By default, remove ldif.done files after running cl-dump.
Add an option '-l' which allows keep the files.
Modify 'dsconf replication dump-changelog' command accordingly.
Update man files.

https://pagure.io/389-ds-base/issue/50572

Reviewed by: ?

rebased onto 2123d9f33e1cdcc90d58570510e117e63dc647f9
4 years ago

Never use negatives in options, because this confuses the logic. It should be "remove_ldif_done=True" and then the override is to set false. IIRC this is in the style guide :)

Nack, this has to be a positive option IE "preserve-cl-ldif". Don't have negatives in cli options,

rebased onto ba2bfea1c35a2767f7d145782c0270e7c0d37f2f
4 years ago

Nack, this has to be a positive option IE "preserve-cl-ldif". Don't have negatives in cli options,

Right, missed that. Thank you! Fixed

Indentation is code in this code block sequence :-)

LGTM besides that one minor comment

Ack from me too.

Pull-Request has been merged by spichugi
4 years ago

389-ds-base is moving from Pagure to Github. This means that new issues and pull requests
will be accepted only in 389-ds-base's github repository.

This pull request has been cloned to Github as issue and is available here:
- https://github.com/389ds/389-ds-base/issues/3629

If you want to continue to work on the PR, please navigate to the github issue,
download the patch from the attachments and file a new pull request.

Thank you for understanding. We apologize for all inconvenience.

Pull-Request has been closed by spichugi
3 years ago
Metadata
None
No Tags
Changes Summary 5
+15 -8
file changed
ldap/admin/src/scripts/cl-dump.pl
+7 -4
file changed
man/man1/cl-dump.1
+7 -4
file changed
man/man1/cl-dump.pl.1
+5 -1
file changed
src/lib389/lib389/cli_conf/replication.py
+7 -2
file changed
src/lib389/lib389/replica.py
Powered by Pagure 5.13.3
Documentation • File an Issue • About • SSH Hostkey/Fingerprint
© Red Hat, Inc. and others.