Go Back   DaemonForums > Miscellaneous > Programming

Programming C, bash, Python, Perl, PHP, Java, you name it.

Thread Tools Display Modes
Prev Previous Post   Next Post Next
  #1   (View Single Post)  
Old 9th February 2014
J65nko J65nko is offline
Join Date: May 2008
Location: Budel - the Netherlands
Posts: 3,597
Default Simplifying pflog file parsing

Last week I decided it was about time to write a Perl script for parsing /var/log/pflog files.

If you use logging in the pf.conf ruleset, the logged packets are written to this file in a modified tcpdump PCAP format.

You can read this file with something like # tcpdump -tttt -nr /var/log/pflog | less or save it in a text file with # tcpdump -tttt -nr /var/log/pflog > ./pflog.txt.

The output looks like:
#   date(1)    time(2)       ty  source(4)      destination(5)    header info (6)
# 2014-01-31 16:14:30.938665 IP > ICMP echo request, id 0, seq 0, length 64
# 2014-01-31 16:16:35.262293 IP > Flags [S], seq 1118489574, win 14600, options [mss 1460,sackOK,TS[|tcp]>
To parse this with Perl I came up with the following subroutine using a long regular expression:
        if ( /^                         # at beginning of line
            (\d{4}-\d{2}-\d{2})         # capture date yyyy-mm-dd
            \s+                         # one or more spaces
            (\d{2}:\d{2}:\d{2}\.\d+)    # hh:mm:sec.fractional
            (\w+)                       # traffic type i.e. IP
            (\d+\.\d+.\d+\.\d+          # source IP
                (?:                     # non-capture grouping start
                \.\d+                   # a '.' and one of more digits as port number
                )?                      # end of optional port number
            )                           # end capture
            (\d+\.\d+.\d+\.\d+          # destination IP address
                (?:                     # non-capture grouping start
                \.\d+                   # optional port number
                )?                      # end of optional port number
            )                           # end capture
            (.+)                        # packet header info
            $                           # up to end of line
            /x                          # allow comments and whitespace

After two or three tries it worked. Although it looks complicated, it actually is quite straightforward.
The only tricky thing was to use the non-capturing grouping (?: ........ ) to match an optional port number.
tcpdump prefixes the port number with a period and appends it to the IP address. In > the source IP is using source port 55459, while the destination IP is with port 25.

After a few days, after a walk with the dog, I found a much simpler method. Well actually two.
You don't need to be a genius to debug a pf.conf firewall ruleset, you just need the guts to run tcpdump

Last edited by J65nko; 9th February 2014 at 06:33 PM.
Reply With Quote

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Keeping your /var/log/pflog file clean and managable J65nko Guides 3 8th February 2014 11:07 PM
Log Parsing Not Working plexter OpenBSD Packages and Ports 9 16th July 2011 03:37 PM
Parsing emails with 'awk' and 'perl' J65nko Guides 1 24th February 2011 03:34 AM
Why PFLOG can't LOG anything????? chamnanpol FreeBSD General 1 18th June 2008 07:09 PM

All times are GMT. The time now is 09:01 AM.

Powered by vBulletin® Version 3.8.4
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.
Content copyright © 2007-2010, the authors
Daemon image copyright ©1988, Marshall Kirk McKusick