Not sure why, but using logconv.pl -V with DB_File creates temporary DB files in /var/tmp called BDBxxxxx. These can grow to be very, very large. They need to go in the -D $dataLocation directory.
It appears these BDB files in /var/tmp are completely controlled by the bdb library. I tried to see if I could flush them by periodically "untie"ing the files and reopening them. While this does clear out those files, as soon as we "tie" the files again and make updates, those BDB files reappear. Their size remains the same as it was when they were untied, and then they continue to grow as new updates come in.
Are the BDB files in /var/tmp created for the tied hashes, tied arrays, or both? Is there any way to make them use something other than /var/tmp? We may have to skip using tied hashes/arrays and just use the DB_File bdb interface directly . . .
Replying to [comment:5 rmeggins]:
Are the BDB files in /var/tmp created for the tied hashes, tied arrays, or both?
Actually I have only seen 12 files max in my testing. Your test machine doesn't appear to be around anymore. So initially it looks like this is only for the hashes(16 hashes possible)? Do you recall how many you saw in your testing? Also, I do not know if it's the hashes, or simply once the DB file gets past a certain size that these other files are created.
Is there any way to make them use something other than /var/tmp?
Not that I could find; not using the perl interface at least.
We may have to skip using tied hashes/arrays and just use the DB_File bdb interface directly . . .
Maybe, but it's possible it might still create these files.
reassigning to Rich since he is working on the fix.
logconv wip - logconv.pl -V is about 30 times faster logconv.pl
ugh - need to clean up formatting
0001-Ticket-47501-logconv.pl-uses-var-tmp-for-BDB-temp-fi.patch 0001-Ticket-47501-logconv.pl-uses-var-tmp-for-BDB-temp-fi.patch
same patch but ignore whitespace diffs ignore-ws.patch
Looks good, except can you bump the version number at the top of the script please? Otherwise, ack.
30 times faster! It's amazing how much the array/DB_RECNO had an impact on performance.
To ssh://git.fedorahosted.org/git/389/ds.git e6eab21..4d20922 master -> master commit 4d20922 Author: Rich Megginson rmeggins@redhat.com Date: Tue Jul 23 17:05:32 2013 -0600
Ticket has been cloned to Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1013163
To ssh://git.fedorahosted.org/git/389/ds.git fbece32..313dd8e 389-ds-base-1.2.11 -> 389-ds-base-1.2.11 commit e53b8ea Author: Rich Megginson rmeggins@redhat.com Date: Tue Jul 23 17:05:32 2013 -0600
To ssh://git.fedorahosted.org/git/389/ds.git c96eaa0..9103b3e 389-ds-base-1.3.0 -> 389-ds-base-1.3.0 commit 223c053 Author: Rich Megginson rmeggins@redhat.com Date: Tue Jul 23 17:05:32 2013 -0600
To ssh://git.fedorahosted.org/git/389/ds.git ff9a292..e38685b 389-ds-base-1.3.1 -> 389-ds-base-1.3.1 commit d890e65 Author: Rich Megginson rmeggins@redhat.com Date: Tue Jul 23 17:05:32 2013 -0600
Metadata Update from @mreynolds: - Issue assigned to rmeggins - Issue set to the milestone: 1.3.3 - 10/13 (October)
389-ds-base is moving from Pagure to Github. This means that new issues and pull requests will be accepted only in 389-ds-base's github repository.
This issue has been cloned to Github and is available here: - https://github.com/389ds/389-ds-base/issues/838
If you want to receive further updates on the issue, please navigate to the github issue and click on subscribe button.
subscribe
Thank you for understanding. We apologize for all inconvenience.
Metadata Update from @spichugi: - Issue close_status updated to: wontfix (was: Fixed)
Login to comment on this ticket.