#47910 allow logconv.pl -S/-E switches to work even when timestamps not present in access log
Closed: Fixed None Opened 4 years ago by rmeggins.

Ticket was cloned from Red Hat Bugzilla (product Fedora): Bug 1145948

Created attachment 940690
access log

Description of problem:
Attached is a small example access log which can be used to reproduce this, I'm
seeing the same on F20 & RHEL6.

$ logconv.pl ./access - reports meaningful statistics

$ logconv.pl -S foo ./access - reports all zeroes but does not complain about
the invalid time string - should probably error out if the time string is
invalid as otherwise the results will be unexpected and it's hard to spot such
mistakes

$ logconv.pl -S '[24/Sep/2014:10:10:00 +0300]' ./access - reports all zeroes
and the printed "Start of Logs" does not match the provided time string

Using -E does not seem to have any effect.

Version-Release number of selected component (if applicable):
F20 / RHEL6

{{{
1632 if ($currentTimeInSecs > $endTimeInSecs) {
}}}

Shouldn't this be $currentTimeInSecs <= $endTimeInSecs ?

Replying to [comment:5 rmeggins]:

{{{
1632 if ($currentTimeInSecs > $endTimeInSecs) {
}}}

Shouldn't this be $currentTimeInSecs <= $endTimeInSecs ?

Nope. We want to stop processing the log after we have passed the end time.

so how much slower is processing now that we have to convert every timestamp?

Replying to [comment:8 rmeggins]:

so how much slower is processing now that we have to convert every timestamp?

This was a concern I had, but I would think that when using a start/stop time you would not be processing gigs of logs, but smaller snipets.

So I tested 3 million lines of logs:

With start/end times: 59 seconds
WIthout start/end times: 47 seconds

Looks like there is about a 25% performance hit.

If someone is using start/end times to troubleshoot a particualr issue, then I think this is fine. But I'm not sure who asked for this feature in the first place, and in what capacity it was going to be used.

bb55a72..b0213c1 master -> master
commit b0213c1
Author: Mark Reynolds mreynolds@redhat.com
Date: Wed Jun 10 10:49:59 2015 -0400

QE found issue with date parsing with invalid values...

2c484cc..8495afa master -> master
commit 8495afa
Author: Mark Reynolds mreynolds@redhat.com
Date: Tue Jul 14 10:09:00 2015 -0400

69fd1f1..3bf1daa 389-ds-base-1.3.4 -> 389-ds-base-1.3.4
commit 3bf1daa

QE found an issue when the start time is greater than the end time. Reopening...

eb3086d..34ffa6c master -> master
commit 34ffa6c
Author: Mark Reynolds mreynolds@redhat.com
Date: Mon Jul 20 11:18:12 2015 -0400

23a3ff6..16b95b1 389-ds-base-1.3.4 -> 389-ds-base-1.3.4
commit 16b95b1

Test case added for - allow logconv.pl -S/-E switches to work even when timestamps not present in access log
0001-Ticket-47910-allow-logconv.pl-S-E-switches-to-work-e.2.patch

391acfc..71987ed master -> master
commit 71987ed
Author: amsharma amsharma@redhat.com
Date: Fri Aug 7 17:55:20 2015 +0530

096b386..73d7dcf 389-ds-base-1.3.4 -> 389-ds-base-1.3.4
commit 73d7dcf
Author: amsharma amsharma@redhat.com
Date: Fri Aug 7 17:55:20 2015 +0530

Metadata Update from @mreynolds:
- Issue assigned to mreynolds
- Issue set to the milestone: 1.3.4.0

2 years ago

Login to comment on this ticket.

Metadata