|
Programming C, bash, Python, Perl, PHP, Java, you name it. |
|
Thread Tools | Display Modes |
|
|||
Simplifying pflog file parsing
Last week I decided it was about time to write a Perl script for parsing /var/log/pflog files.
If you use logging in the pf.conf ruleset, the logged packets are written to this file in a modified tcpdump PCAP format. You can read this file with something like # tcpdump -tttt -nr /var/log/pflog | less or save it in a text file with # tcpdump -tttt -nr /var/log/pflog > ./pflog.txt .The output looks like: Code:
# date(1) time(2) ty source(4) destination(5) header info (6) # 2014-01-31 16:14:30.938665 IP 80.25.124.114 > 1.2.22.222: ICMP echo request, id 0, seq 0, length 64 # 2014-01-31 16:16:35.262293 IP 180.188.194.9.55459 > 1.2.22.222.25: Flags [S], seq 1118489574, win 14600, options [mss 1460,sackOK,TS[|tcp]> Code:
if ( /^ # at beginning of line (\d{4}-\d{2}-\d{2}) # capture date yyyy-mm-dd \s+ # one or more spaces (\d{2}:\d{2}:\d{2}\.\d+) # hh:mm:sec.fractional \s+ (\w+) # traffic type i.e. IP \s+ (\d+\.\d+.\d+\.\d+ # source IP (?: # non-capture grouping start \.\d+ # a '.' and one of more digits as port number )? # end of optional port number ) # end capture \s+ > \s+ (\d+\.\d+.\d+\.\d+ # destination IP address (?: # non-capture grouping start \.\d+ # optional port number )? # end of optional port number ) # end capture :\s+ (.+) # packet header info $ # up to end of line /x # allow comments and whitespace ) The only tricky thing was to use the non-capturing grouping (?: ........ ) to match an optional port number. tcpdump prefixes the port number with a period and appends it to the IP address. In 180.188.194.9.55459 > 1.2.22.222.25 the source IP is 180.188.194.9 using source port 55459, while the destination IP is 1.2.22.222 with port 25. After a few days, after a walk with the dog, I found a much simpler method. Well actually two.
__________________
You don't need to be a genius to debug a pf.conf firewall ruleset, you just need the guts to run tcpdump Last edited by J65nko; 9th February 2014 at 06:33 PM. |
|
|||
Simplification nr 1
Code:
# date(1) time(2) ty source(4) destination(5) header info (6) # 2014-01-31 16:14:38.108331 IP 80.34.103.251 > 1.2.22.222: ICMP echo request [snip] ^ A second [b]split] on a space or blank then separates the first part into the fields that I am interested in. Code:
sub read_simple { my ($leading, $info); my ($date, $time, $type, $source, $direction, $dest); while (<DATA>) { ($leading, $info) = split(/: /); print "Leading: ", $leading, "\n"; print "Info : ", $info, "\n"; ($date, $time, $type, $source, $direction, $dest) = split( / /, $leading); show_raw($date, $time, $type, $source, $dest, $info); export( $date, $time, $type, $source, $dest, $info); } }
__________________
You don't need to be a genius to debug a pf.conf firewall ruleset, you just need the guts to run tcpdump Last edited by J65nko; 9th February 2014 at 05:58 PM. |
|
|||
Simplification nr 2
According the documention of the split function you can limit the number of fields that will be separated or split. From http://perldoc.perl.org/functions/split.html :
Quote:
Code:
# date(1) time(2) ty source(4) destination(5) header info (6) # 2014-01-31 16:14:30.938665 IP 80.25.124.114 > 1.2.22.222: ICMP echo request, id 0, seq 0, length 64 1 2 3 4 5 6 7 Code:
sub read_simple { my ($leading, $info); my ($date, $time, $type, $source, $direction, $dest); while (<DATA>) { ($date, $time, $type, $source, $direction, $dest, $info) = split( / /, $_, 7); show_raw($date, $time, $type, $source, $dest, $info); #export( $date, $time, $type, $source, $dest, $info); } } Code:
#!/usr/bin/perl use warnings; use strict; my $NULL = '\N'; # MySQL null value # date(1) time(2) ty source(4) destination(5) header info (6) # 2014-01-31 16:14:30.938665 IP 80.25.124.114 > 1.2.22.222: ICMP echo request, id 0, seq 0, length 64 # 2014-01-31 16:16:35.262293 IP 180.188.194.9.55459 > 1.2.22.222.25: Flags [S], seq 1118489574, win 14600, options [mss 1460,sackOK,TS[|tcp]> sub show_raw { my ($date, $time, $type, $source, $dest, $info) = @_; print STDERR <<END; Date : $date Time : $time Type : $type Source IP : $source Destination IP : $dest Info : $info ============================= END } sub read_simple { my ($leading, $info); my ($date, $time, $type, $source, $direction, $dest); while (<DATA>) { ($date, $time, $type, $source, $direction, $dest, $info) = split( / /, $_, 7); show_raw($date, $time, $type, $source, $dest, $info); #export( $date, $time, $type, $source, $info); } } read_simple(); __END__ 2014-01-31 16:14:30.938665 IP 80.25.124.114 > 1.2.22.222: ICMP echo request, id 0, seq 0, length 64 2014-01-31 16:16:35.262293 IP 180.188.194.9.55459 > 1.2.22.222.25: Flags [S], seq 1118489574, win 14600, options [mss 1460,sackOK,TS[|tcp]> 2014-01-31 16:16:38.260924 IP 180.188.194.9.55459 > 1.2.22.222.25: Flags [S], seq 1118489574, win 14600, options [mss 1460,sackOK,TS[|tcp]> Code:
Why did I think I had to go that way?
__________________
You don't need to be a genius to debug a pf.conf firewall ruleset, you just need the guts to run tcpdump Last edited by J65nko; 9th February 2014 at 06:35 PM. |
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
Keeping your /var/log/pflog file clean and managable | J65nko | Guides | 3 | 8th February 2014 11:07 PM |
Log Parsing Not Working | plexter | OpenBSD Packages and Ports | 9 | 16th July 2011 03:37 PM |
Parsing emails with 'awk' and 'perl' | J65nko | Guides | 1 | 24th February 2011 03:34 AM |
Why PFLOG can't LOG anything????? | chamnanpol | FreeBSD General | 1 | 18th June 2008 07:09 PM |