Netflow accounting using pflow

[dunkel]

Hi folks,

I plan to move our core routers from FreeBSD to OpenBSD. Currently I use netgraph and ng_netflow on the FreeBSD machines for netflow accounting.

As there is a netflow kernel implentation with pflow in OpenBSD, too, I decided to use this.
Unfortunately I seems that flows are exported twice for each connection.

A quick example:

I have a machine that collects the netflows:

deb-01:~# flow-receive 0.0.0.0/0.0.0.0/2100 -o bsdflow
flow-receive: setsockopt(size=4194304)

flow-receive: New exporter: time=1294143869 src_ip=192.168.89.150 dst_ip=192.168.89.52 d_version=5

I have a router using pflow between two machines which I use to do performance tests:

rtr-fra-01# cat /etc/hostname.pflow0
flowsrc 192.168.89.150 flowdst 192.168.89.52:2100
rtr-fra-01# grep -v ^\# /etc/pf.conf 
set skip on lo
pass keep state (pflow)         # to establish keep-state
block in on ! lo0 proto tcp to port 6000:6010
rtr-fra-01#

Now I generate some packets that are forwarded by the router:

grabnebel:~# nuttcp -i1 192.168.92.55
  102.0721 MB /   1.00 sec =  856.2173 Mbps
  111.3174 MB /   1.00 sec =  933.8174 Mbps
  111.6529 MB /   1.00 sec =  936.6071 Mbps
  111.6529 MB /   1.00 sec =  936.5912 Mbps
  111.6529 MB /   1.00 sec =  936.6240 Mbps
  111.6529 MB /   1.00 sec =  936.6287 Mbps
  111.6571 MB /   1.00 sec =  936.6241 Mbps
  111.6529 MB /   1.00 sec =  936.6296 Mbps
  111.6598 MB /   1.00 sec =  936.6725 Mbps
  111.6529 MB /   1.00 sec =  936.5959 Mbps

 1108.1875 MB /  10.01 sec =  928.3151 Mbps 6 %TX 20 %RX
grabnebel:~#

In pf's state table I see two records - one for each direction of the connection.

rtr-fra-01# pfctl -vss | grep -A2 5001 
all tcp 192.168.92.55:5001 <- 192.168.89.55:50609       FIN_WAIT_2:FIN_WAIT_2
   [3659736762 + 5888] wscale 6  [491808162 + 1749184] wscale 6
   age 00:00:39, expires in 00:01:01, 802503:402038 pkts, 1203748980:20916988 bytes, rule 0, pflow
all tcp 192.168.89.55:50609 -> 192.168.92.55:5001       FIN_WAIT_2:FIN_WAIT_2
   [491808162 + 1749184] wscale 6  [3659736762 + 5888] wscale 6
   age 00:00:39, expires in 00:01:01, 802503:402038 pkts, 1203748980:20916988 bytes, rule 0, pflow
rtr-fra-01#

After the flows have been exported I can see them on the accounting machine:

deb-01:~# flow-print < bsdflow | grep 5001
192.168.89.55    192.168.92.55    6     50609    5001     1203748980  802503    
192.168.92.55    192.168.89.55    6     5001     50609    20916988    402038    
192.168.89.55    192.168.92.55    6     50609    5001     1203748980  802503    
192.168.92.55    192.168.89.55    6     5001     50609    20916988    402038    
deb-01:~#

So it looks like pflow is exporting both records via netflow. But unfortunately it seems that both RX and TX traffic data are exported for each record which results in redundant accounting data.

Any suggestions?

Regards,
Bernd

[dunkel]

Ok. Obviously there are states for each interface the traffic passes.
Restricting the pflow flag to only outgoing traffic solves the problem.

pass out keep state (pflow)