Firewall Blocking Good Traffic

[plexter]

Hello,

First a quick text diagram.

{internet} <--> [PIX Firewall] <--> [OpenBSD PF] <--> [rest of internal net]

I believe my problem is with my PF firewall rules however if you feel otherwise I will investigate further on the PIX.

What is happening is that while PF is enabled. (works after disabling it)

I see the following (as an example) in my logs on the PIX:

6|Jan 01 2009|19:24:05|106015|10.10.200.3|CENSORED-IP|Deny TCP (no connection) from 10.10.200.3/12234 to CENSORED-IP/110 flags FIN ACK  on interface inside
6|Jan 01 2009|19:21:58|106015|10.10.200.3|CENSORED-IP|Deny TCP (no connection) from 10.10.200.3/12235 to CENSORED-IP/80 flags ACK  on interface inside
6|Jan 01 2009|19:21:48|106015|10.10.200.3|CENSORED-IP|Deny TCP (no connection) from 10.10.200.3/12235 to CENSORED-IP/80 flags RST ACK  on interface inside
6|Jan 01 2009|19:21:39|106015|10.10.200.3|CENSORED-IP|Deny TCP (no connection) from 10.10.200.3/12234 to CENSORED-IP/110 flags RST ACK  on interface inside
6|Jan 01 2009|19:21:10|106015|10.10.200.3|CENSORED-IP|Deny TCP (no connection) from 10.10.200.3/12229 to CENSORED-IP/80 flags ACK  on interface inside
6|Jan 01 2009|19:20:55|106015|10.10.200.3|CENSORED-IP|Deny TCP (no connection) from 10.10.200.3/12229 to CENSORED-IP/80 flags RST ACK  on interface inside

The connections are for port 80 which is software trying to get updates for itself. Also port 110 which is a connection to an external mail server.

PF Configuration:

#Last Modified Dec 29 2008

#Interfaces
ext_if="fxp0"
int_if="fxp1"

#Devices

dlbox = "{ 10.10.200.4 }"
main = "{ 10.10.200.13 }"
ftpsrv = "{ 192.168.233.3 }"
websrv = "{ 192.168.233.4 }"
exc2k3 = "{ 10.10.200.3 }"
voipnet = "{ 192.168.255.0/24 }"
sshport = "{ 2201 }"

#Options
set require-order yes
set block-policy drop
set loginterface $ext_if
set loginterface $int_if
set state-policy floating
set fingerprints "/etc/pf.os"
set ruleset-optimization none

#Timeout Settings
set optimization aggressive
set timeout { frag 10, tcp.established 3600 }
set timeout { tcp.first 30, tcp.closing 10, tcp.closed 10, tcp.finwait 10 }
set timeout { udp.first 30, udp.single 30, udp.multiple 30 }
set timeout { other.first 30, other.single 30, other.multiple 30 }
set timeout { adaptive.start 5000, adaptive.end 10000 }

#Connection Normalization 

scrub all random-id min-ttl 254 max-mss 1452 reassemble tcp fragment reassemble

#Queuing ALTQ
#Upload assumed aprox 80kb or 640Kb queued aprox 94%

altq on $ext_if cbq bandwidth 600Kb queue { tcpack, voip, dns, mainpc, webftp, dlbox, std }

queue std bandwidth 10% priority 1 cbq(borrow red default)
queue dlbox bandwidth 10% priority 2 cbq(borrow red)
queue webftp bandwidth 5% priority 3 cbq(borrow red)
queue mainpc bandwidth 10% priority 4 cbq(borrow red)
queue dns bandwidth 5% priority 5 cbq(borrow red)
queue tcpack bandwidth 20% priority 6 cbq(borrow red)
queue voip bandwidth 40% priority 7 cbq(borrow red) 

block in log on $ext_if

#TCP Traffic Out on $ext_if
pass out on $ext_if inet proto tcp from any to any modulate state queue(std, tcpack)
pass out quick on $ext_if inet proto tcp from $dlbox to any modulate state queue(dlbox, tcpack)
pass out quick on $ext_if inet proto tcp from $ftpsrv to any port ftp synproxy state queue(webftp, tcpack)
pass out quick on $ext_if inet proto tcp from $websrv to any port 10001 synproxy state queue(webftp, tcpack)
pass out quick on $ext_if inet proto tcp from $exc2k3 to any synproxy state queue(webftp, tcpack)
pass out quick on $ext_if inet proto tcp from $main to any modulate state queue(mainpc, tcpack)
pass out quick on $ext_if inet proto tcp from any to any port domain modulate state queue(dns, tcpack)

pass out quick on $ext_if inet proto tcp from $main to any port 63600  modulate state queue(dlbox, tcpack) 

#UDP Traffic Out on $ext_if
#pass out on $ext_if inet proto udp from any to any keep state queue(std, tcpack)
pass out quick on $ext_if inet proto udp from $main to any keep state queue(mainpc, tcpack)
pass out quick on $ext_if inet proto udp from any to any port ntp keep state queue(dns, tcpack)
pass out quick on $ext_if inet proto udp from any to any port domain keep state queue(dns, tcpack)
pass out quick on $ext_if inet proto udp from any to any port 514 queue(std, tcpack)
pass out log quick on $ext_if inet proto udp from any to any port 5060 keep state queue(voip, tcpack)
pass out log quick on $ext_if inet proto udp from any to any port 5000 keep state queue(voip, tcpack)
pass out log quick on $ext_if inet from $voipnet to any keep state queue(voip, tcpack)
pass out quick on $ext_if inet proto udp from any to any port 63600 keep state queue(dlbox, tcpack)

#TCP/UDP Traffic Out on $ext_if

#TCP Traffic In on $ext_if

pass in on $ext_if inet proto tcp from any to any port 63600 modulate state queue(dlbox, tcpack)
pass in quick on $ext_if inet proto tcp from any to any port 10001 modulate state

#UDP Traffic In on $ext_if

pass in on $ext_if inet proto udp from any to any port domain keep state queue(dns, tcpack)
pass in on $ext_if inet proto udp from any to any port 514 queue(std, tcpack)
pass in on $ext_if inet proto udp from any to any port 63600 keep state queue(dlbox, tcpack)
pass in log on $ext_if inet proto udp from any to any port 5060 keep state queue(voip, tcpack)
pass in log on $ext_if inet proto udp from any to any port 5000 keep state queue(voip, tcpack)

Hope someone can help!
Thanks a bunch!

[DutchDaemon]

Quick suggestions:

Use 'modulate state' on outbound TCP connections, use 'synproxy state' on inbound TCP connections destined for your LAN , and use 'keep state' on inbound TCP (to the firewall itself), in/outbound UDP and in/outbound ICMP.

Do not use things like queue(a, b) on non-TCP traffic. UDP and ICMP have no TCP acks, so this might confuse altq or have undesirable side-effects (like filling up the wrong queues).

[s0xxx]

I'm not familiar with Cisco's PIX, but if you suspect pf is blocking some connections then make sure you have log keyword in either block or pass rules and run tcpdump(8) on pflog(4) interface or check its log file in /var/log/pflog. More about that in: http://www.openbsd.org/faq/pf/logging.html

[plexter]

Hello there,

Thanks for your replies.

I have adjusted my pf.conf file with your state suggestions. This seems to have actually helped. Ports are no longer being blocked Thanks!

However I with queues (a, b) ; the first field (a) is it not just a general marker where the second (b) is for TOS which would account for more than TCP ack?

I have adjusted the queues as you suggested anyway. I have not experienced any difference so far so I suppose it is possible (b) was not doing anything at all. Clarification would be nice if possible.

In regards to logging. I already did have log setup on some rules. However since I made the above changes I no longer see any log results for INCOMING traffic. I'm not sure why or how this happened. Perhaps I made a typo of some kind?

#Last Modified Jan 03 2009

#Interfaces
ext_if="fxp0"
int_if="fxp1"

#Devices
dlbox = "{ 10.10.200.4 }"
main = "{ 10.10.200.13 }"
ftpsrv = "{ 192.168.233.3 }"
websrv = "{ 192.168.233.4 }"
exc2k3 = "{ 10.10.200.3 }"
voipnet = "{ 192.168.255.0/24 }"
voipone = "{ 192.168.255.2 }"
sshport = "{ 2201 }"
VOIPSERV = "{ CENSORED }"

#Options
set require-order yes
set block-policy drop
set loginterface $ext_if
set loginterface $int_if
set state-policy floating
set fingerprints "/etc/pf.os"
set ruleset-optimization none

#Timeout Settings
set optimization aggressive
set timeout { frag 10, tcp.established 3600 }
set timeout { tcp.first 30, tcp.closing 10, tcp.closed 10, tcp.finwait 10 }
set timeout { udp.first 30, udp.single 30, udp.multiple 30 }
set timeout { other.first 30, other.single 30, other.multiple 30 }
set timeout { adaptive.start 5000, adaptive.end 10000 }

#Connection Normalization

scrub all random-id min-ttl 254 max-mss 1452 reassemble tcp fragment reassemble

#Queuing ALTQ
#Upload assumed aprox 80kb or 640Kb queued aprox 94%

altq on $ext_if cbq bandwidth 600Kb queue { tcpack, voip, dns, mainpc, webftp, dlbox, std }

queue std bandwidth 10% priority 1 cbq(borrow red default)
queue dlbox bandwidth 10% priority 2 cbq(borrow red)
queue webftp bandwidth 5% priority 3 cbq(borrow red)
queue mainpc bandwidth 10% priority 4 cbq(borrow red)
queue dns bandwidth 5% priority 5 cbq(borrow red)
queue tcpack bandwidth 20% priority 6 cbq(borrow red)
queue voip bandwidth 40% priority 7 cbq(borrow red)

block in log on $ext_if

#TCP Traffic Out on $ext_if
pass out on $ext_if inet proto tcp from any to any modulate state queue(std, tcpack)
pass out quick on $ext_if inet proto tcp from $dlbox to any modulate state queue(dlbox, tcpack)
pass out quick on $ext_if inet proto tcp from $ftpsrv to any port ftp modulate state queue(webftp, tcpack)
pass out quick on $ext_if inet proto tcp from $websrv to any port 10001 modulate state queue(webftp, tcpack)
pass out quick on $ext_if inet proto tcp from $exc2k3 to any modulate state queue(webftp, tcpack)
pass out quick on $ext_if inet proto tcp from $main to any modulate state queue(mainpc, tcpack)
pass out quick on $ext_if inet proto tcp from any to any port domain modulate state queue(dns, tcpack)


#UDP Traffic Out on $ext_if
#pass out on $ext_if inet proto udp from any to any keep state queue(std, tcpack)
pass out quick on $ext_if inet proto udp from any to any port ntp queue(dns)
pass out quick on $ext_if inet proto udp from any to any port domain queue(dns)
pass out quick on $ext_if inet proto udp from any to any port 514 queue(std)
pass out quick log on $ext_if inet proto udp from $voipone to $VOIPSERV port 5060 keep state queue(voip) label VOIP_OUT
pass out quick log on $ext_if inet proto udp from $voipone to $VOIPSERV port 5000 keep state queue(voip) label VOIP_OUT
pass out quick log on $ext_if inet proto udp from $voipone to $VOIPSERV port > 10000 keep state queue(voip) label VOIP_OUT
pass out on $ext_if inet proto udp from $dlbox to any port 63600 keep state queue(dlbox)
#TCP/UDP Traffic Out on $ext_if

#TCP Traffic In on $ext_if

pass in quick on $ext_if inet proto tcp from any to any port 10001 synproxy state queue(webftp, tcpack)

#UDP Traffic In on $ext_if

pass in on $ext_if inet proto udp from any to any port domain queue(dns)
pass in on $ext_if inet proto udp from $voipone to any port ntp queue(dns)
pass in on $ext_if inet proto udp from any to any port 514 queue(std)
pass in on $ext_if inet proto udp from any to $dlbox port 63600 keep state queue(dlbox)
pass in quick log on $ext_if inet proto udp from any to $voipone port 5060 keep state queue(voip) label VOIP_IN
pass in quick log on $ext_if inet proto udp from any to $voipone port 5000 keep state queue(voip) label VOIP_IN
pass in quick log on $ext_if inet proto udp from any to $voipone port > 10000 queue(voip) label VOIP_IN

Any idea?

Thanks.

[DutchDaemon]

Based on the manual, you're right about the lowdelay stuff. I guess it depends on how you design your queues. I usually put stuff that performs badly under congestion (like voip) in its own high-priority queue. Anyway, you can see how (and if) your queues perform by installing pftop and pressing '8' after starting it (you can also use pfctl -sq -vv, but that's less granular and harder to read).

[jggimi]

systat queues or states may be helpful, also.

[plexter]

I believe I figured out why I do not see the log messages now. Since the traffic is going both ways with state it only logs the initial connection. I changed to log (all) and I see the traffic as expected now.

systat is a neat tool. I had not seen this before. Thanks for sharing!


Anyway I believe my initial problem has been resolved. Thanks all for your help/input. Very much appreciated.