[Solved-3 Solutions] How to Filter log file entries based on date range ? - linux server - linux pc
Problem:
How to Filter log file entries based on date range ?
Solution 1:
- Assuming "HH:MM" to show, and to use awk:
awk -v start=00:00 -v stop=09:00 'start <= $2 && $2 < stop' file.log- This one is terrible: what date is "09/10/11"?
Assuming this it "YY/MM/DD"
awk -v date="$(date +%y/%m/%d)" \
-v start=00:00:00 \
-v stop=09:00:00 \
-v search="File format not found" \
'$1 == date && start <= $2 && $2 < stop && $0 ~ search' file.logSolution 2:
#!/usr/bin/perl -ws
# This script parse logfiles for a specific period of time
sub usage
{
printf "Usage: %s -s=<start time> [-e=<end time>] <logfile>\n";
die $_[0] if $_[0];
exit 0;
}
use Date::Parse;
usage "No start time submited" unless $s;
my $startim=str2time($s) or die;
my $endtim=str2time($e) if $e;
$endtim=time() unless $e;
usage "Logfile not submited" unless $ARGV[0];
open my $in, "<" . $ARGV[0] or usage "Can't open '$ARGV[0]' for reading";
$_=<$in>;
exit unless $_; # empty file
# Determining regular expression, depending on log format
my $logre=qr{^(\S{3}\s+\d{1,2}\s+(\d{2}:){2}\d+)};
$logre=qr{^[^\[]*\[(\d+/\S+/(\d+:){3}\d+\s\+\d+)\]} unless /$logre/;
while (<$in>)
{
/$logre/ && do
{
my $ltim=str2time($1);
print if $endtim >= $ltim && $ltim >= $startim;
};
};Using this code:
./timelapsinlog.pl -s=08:18 -e=08:24 /path/to/logfileFor printing logs between 08h18 and 08h24
./timelapsinlog.pl -s='2018/10/20 08:18:12' /path/to/logfileFor printing from oct 20th, 8h18'12" upto now.
In order to reduce perl -s switch to permit auto-assignement of variables from commandline: -s=08:18 will populate a variable $s which will have 0:18.
Note: This hold two different kind of regex for two different log standard. Different date/time format parsing, either post the own regex or post a sample of formatted date from logfile.
^(\S{3}\s+\d{1,2}\s+(\d{2}:){2}\d+) # ^Jan 1 01:23:45
^[^\[]*\[(\d+/\S+/(\d+:){3}\d+\s\+\d+)\] # ^... [01/Jan/2017:01:23:45 +0000]Solution 3:
awk: invalid -v option, is an active IPs in a predefined time range to get by script:
cat <FILE_NAME> | awk '$4 >= "[04/Jul/2017:07:00:00" && $4 < "[04/Jul/2017:08:00:00"' | awk '{print $1}' | sort -n | uniq -c | sort -nr | head -20