DaemonForums  

Go Back   DaemonForums > FreeBSD > FreeBSD General

FreeBSD General Other questions regarding FreeBSD which do not fit in any of the categories below.

Reply
 
Thread Tools Display Modes
  #1   (View Single Post)  
Old 1st December 2008
ivanatora ivanatora is offline
Real Name: Ivan
Fdisk Soldier
 
Join Date: Jul 2008
Location: Bulgaria
Posts: 51
Thanked 2 Times in 1 Post
Default pf: why is that rule not working?

Hello,
The situation is simple: two machines are behind NAT and I'm operating on the NAT box. The NAT is set up correctly - both of the machines are connected to the Internet. I have a few IPs from Internet that are put into a table <data>.
I'm trying to learn PF, but something is not going well. I have a rule that doesn't match. In order to debug things, I've set up a logging on that rule and it really doesn't match at all. Could you explain me why?
Forget about the (probably messed up) ALTQ, now everything I want is to understand why the last rule doesn't match.
Code:
### Macros
int_if = "re0"
ext_if = "rl0"
ext_ip = "192.168.1.2"

### Tables
table <network>  { 192.168.0.34, 192.168.0.223 }
table <data> persist file "/root/ip-store.data"

### Normalizations
scrub in all

### Queueing

altq on $int_if hfsc bandwidth 10Mb queue {general, data}
queue general bandwidth 4Mb hfsc (realtime 4Mb upperlimit 4Mb default)   
queue data bandwidth 1Mb hfsc (realtime 128Kb upperlimit 256Kb)

### Translation
nat pass on $ext_if from <network> to any -> $ext_ip

### Filtering

#pass log (all to pflog0) on $ext_if proto icmp # this is working on pflog0 or pflog1, so probability of not working logging devices is zero
pass out log (all to pflog1) on $int_if proto tcp from <data> to <network> #this is not working - nothing is logged to pflog1
First I made myself sure there are some ips into the <data> table - they were there. Then I tried to replace "from <data>" with "from any" - I hought there is no traffic from these hosts, and I tried expanding the rule to apply to the whole traffic. Nothing was logged again.
As you have seen I'm trying to do some ALTQ on the internal interface (for incomming traffic I thing this is the right interface?), and that's why I need that rule to get working. I assume something is totaly wrong in my setup or in my understandings, isn't it?

************************************************** *******************
Things are getting even more confusing!
I changed
Code:
pass out log (all to pflog1) on $int_if proto tcp from <data> to <network>
to
Code:
pass in log (all to pflog1) on $int_if proto tcp from <network> to <data>
and pflog1 began to log!
Despite the "from <network> to any" I see in tcpdump packets flying in both directions, like:
Code:
19:51:36.024411 IP 195.149.248.137.80 > 192.168.0.34.46276:  tcp 1472 [bad hdr length 8 - too short, < 20]
19:51:36.024738 IP 192.168.0.34.46276 > 195.149.248.137.80:  tcp 12 [bad hdr length 8 - too short, < 20]
I thought only the second packet should show up becouse of the "one way" matching only?
And why the opposite direction rule again doesn't match?
Code:
pass out log (all to pflog0) on $int_if proto tcp from <data> to <network>
************************************************** *******************
I'd say there is something interesting even more.
I see packets on pflog1, but according to pfctl -s rules, there shouldn't be any packets at all:
Code:
# pfctl -v -s rules
scrub in all fragment reassemble
  [ Evaluations: 39611     Packets: 19895     Bytes: 7958775     States: 0     ]
  [ Inserted: uid 0 pid 3338 ]
pass out quick on re0 from any to <network> flags S/SA keep state label "incomming"
  [ Evaluations: 5050      Packets: 8         Bytes: 1747        States: 8     ]
  [ Inserted: uid 0 pid 3338 ]
pass in log (all, to pflog1) on re0 proto tcp from <network> to <data> flags S/SA keep state label "??? in"
  [ Evaluations: 4688      Packets: 0         Bytes: 0           States: 0     ]                                                     <--- packets 0 !
  [ Inserted: uid 0 pid 3338 ]
pass in log (all) on rl0 proto tcp from <data> to <network> flags S/SA keep state label "??? out"
  [ Evaluations: 3186      Packets: 0         Bytes: 0           States: 0     ]
  [ Inserted: uid 0 pid 3338 ]

Last edited by ivanatora; 1st December 2008 at 06:57 PM.
Reply With Quote
  #2   (View Single Post)  
Old 1st December 2008
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 3,686
Thanked 214 Times in 189 Posts
Default

The reason you're having trouble is you are confused about NAT. As you probably know already, NAT allows your entire private network to share a single IP address. So, your first failing rule:
Quote:
pass out log (all to pflog1) on $int_if proto tcp from <data> to <network>
fails because there are no packets from <data> to <network>. There never will be. Think about it. Your <data> users do not route packets to IP addresses on <network>. Not directly. They can't. 192.168/16 and the other RFC1918 addresses are not routable on the internet anyway, since they are designed for private networks only.

With NAT, state table tracking is used to keep sessions open when they are originated from within your local network, destined outward. You don't show any rules for that, but if you had them, that is how outbound-originating traffic is managed. Since pass all is the default (well, it is in the OpenBSD implementation of PF, it probably is in FreeBSD) then these are handled by that default rule. "keep state" has been the default for some time, and I assume that's the case with FreeBSD's PF implementation, as you have mentioned success with outbound traffic.

For inbound traffic that needs to reach a server on the internal network, and if a state does not already exist, your NAT router must know about the service and the server. This is commonly done via port forwarding (the "rdr" rules).

Last edited by jggimi; 1st December 2008 at 06:48 PM.
Reply With Quote
  #3   (View Single Post)  
Old 1st December 2008
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 3,686
Thanked 214 Times in 189 Posts
Default

Let me be clear about packets that do originate from <data> IP addresses. The destination IP address will be the IP address of your external interface. That is why you never see packets being matched for that failing rule.
Reply With Quote
  #4   (View Single Post)  
Old 1st December 2008
J65nko J65nko is offline
Administrator
 
Join Date: May 2008
Location: Budel - the Netherlands
Posts: 3,146
Thanked 182 Times in 149 Posts
Default

If you wonder why something is not being logged, just run tcpdump on the interface the traffic is going through.

If I have to debug a firewall with an external and internal NIC, I usually ssh in to the box from my X workstation. In each ssh session I run tcpdump.
  1. On the external interface
    Code:
    tcpdump -ni re0
  2. On the internal interface
    Code:
    # tcpdump -ni bge0 not port ssh
    Note the 'not port ssh' filter, to prevent pollution of the tcpdump output with my own ssh traffic.
  3. On the pflog device
    Code:
    # tcpdump -eni pflog0
__________________
You don't need to be a genius to debug a pf.conf firewall ruleset, you just need the guts to run tcpdump
Reply With Quote
  #5   (View Single Post)  
Old 1st December 2008
s2scott's Avatar
s2scott s2scott is offline
Package Pilot
 
Join Date: May 2008
Location: Toronto, Ontario Canada
Posts: 198
Thanked 9 Times in 8 Posts
Default

Code:
### Translation
nat pass on $ext_if from <network> to any -> $ext_ip
Per the longer explanation already offered, NAT has precedence (occurs first), and, with the pass modifier in effect, no subsequent pass/block RULE is ever hit and, consequently, no pass/block logging is being triggered.

/S

__________________
Never argue with an idiot. They will bring you down to their level and beat you with experience.
Reply With Quote
  #6   (View Single Post)  
Old 1st December 2008
s2scott's Avatar
s2scott s2scott is offline
Package Pilot
 
Join Date: May 2008
Location: Toronto, Ontario Canada
Posts: 198
Thanked 9 Times in 8 Posts
Default

You're looking for something along these lines...

Code:
#STMT 1
nat on $ext_if inet tagged MYIN2OUT -> ($ext_if:0)
#
# STMT 2
pass in log quick on $int_if inet \
 from ($int_if:network) to any \
 tag MYIN2OUT \
 keep state
#
# STMT 3
pass out log quick on $ext_if inet \
 tagged MYIN2OUT \
 keep state
#
In the order of precedence, your traffic originating on your $int_if:network and destined for the internet is FIRST inspected and passed by SMTM#2, then NAT translation occurs by STMT#1, and is then inspected and passed by STMT#3.

The use of tag/tagged means you -- the pf.conf author -- don't need to know the real IP source/dest translations at the point of each statement to write a clean (and leak-proof) pf.conf.


/S
__________________
Never argue with an idiot. They will bring you down to their level and beat you with experience.

Last edited by s2scott; 1st December 2008 at 09:53 PM.
Reply With Quote
  #7   (View Single Post)  
Old 2nd December 2008
ivanatora ivanatora is offline
Real Name: Ivan
Fdisk Soldier
 
Join Date: Jul 2008
Location: Bulgaria
Posts: 51
Thanked 2 Times in 1 Post
Default

J65nko, thanks for your suggestion, but I already have said that I'm using tcpdump and I see there is traffic going trough.

s2scott, this is an interesting way to do NAT in PF. It is not shown into the tutorials, but it works I don't understand what is the SMT3 rule for? It doesn't get matched at all.
And I don't think I can do more evaluations on these packets, because of the 'quick' keyword. AFAIK, when a packet is matched in that rule it doesn't check the next rules at all.

Question: I've accidently added a rule for queuing, and it was working - the download traffic got shaped (did not expect that). Why is it working in the "from <network> to <data>" rule? Isn't that the upload matching traffic rule? To remind - <data> addresses are public servers from Internet.

This is all of my pf.conf:
Code:
### Macros
int_if = "re0"
ext_if = "rl0"
ext_ip = "192.168.1.2"

### Tables
table <network>  { 192.168.0.34, 192.168.0.223 }
table <data> persist file "/root/ip-store.data"

### Normalizations
scrub in all

### Queueing
altq on $int_if hfsc bandwidth 10Mb queue {general, data}
queue general bandwidth 4Mb hfsc (realtime 4Mb upperlimit 4Mb default)   
queue data bandwidth 1Mb hfsc (realtime 128Kb upperlimit 256Kb)

### Translation
nat pass on $ext_if from <network> to any -> $ext_ip

### Filtering
pass in log (all to pflog1) on $int_if proto tcp from <network> to <data> label "??? in" queue data
pass out log (all to pflog0) on $ext_if proto tcp from <data> to <network> label "??? out"
pflog0 still logs nothing, but you have said it is due to 192.168/16 private class network, which is not seen from the Internet. (Btw, I thought PF have states for these connections and it would recognize traffic for the hosts behind NAT)
Reply With Quote
  #8   (View Single Post)  
Old 2nd December 2008
J65nko J65nko is offline
Administrator
 
Join Date: May 2008
Location: Budel - the Netherlands
Posts: 3,146
Thanked 182 Times in 149 Posts
Default

How about adding a log directive to your NAT rule?
__________________
You don't need to be a genius to debug a pf.conf firewall ruleset, you just need the guts to run tcpdump
Reply With Quote
  #9   (View Single Post)  
Old 3rd December 2008
s2scott's Avatar
s2scott s2scott is offline
Package Pilot
 
Join Date: May 2008
Location: Toronto, Ontario Canada
Posts: 198
Thanked 9 Times in 8 Posts
Default

We've tried to explain that there's a big difference between

Code:
nat ...
-and-
Code:
nat pass ...
Code:
nat pass on $ext_if from <network> to any -> $ext_ip
...drop the "pass" and watch what happens.

/S
__________________
Never argue with an idiot. They will bring you down to their level and beat you with experience.
Reply With Quote
Old 3rd December 2008
ivanatora ivanatora is offline
Real Name: Ivan
Fdisk Soldier
 
Join Date: Jul 2008
Location: Bulgaria
Posts: 51
Thanked 2 Times in 1 Post
Default

Quote:
Originally Posted by s2scott View Post
...drop the "pass" and watch what happens.
/S
Oh, didn't saw that I will try that later and repost here.
************************************************** ********************
So, you are proposing this way of alternative doing the NAT (I've applied some queues for testing):
Code:
### Translation
nat on $ext_if tagged DO_NAT -> $ext_ip

### Filtering
pass in quick on $int_if from <network> to <data> tag DO_NAT keep state queue data label "do_nat_data"
pass in quick on $int_if from <network> to any tag DO_NAT keep state label "do_nat"
pass out quick on $ext_if tagged DO_NAT keep state label "revert_nat?"
This works - I'm seeing packets in these labels:
Code:
# pfctl -s label
do_nat_data 450 382 344201 148 6602 234 337599
do_nat 112 226 38383 131 9522 95 28861
revert_nat? 412 608 382584 329 366460 279 16124
I'm still wandering why queueing works for downloads (direction: <data> -> $ext_if -> $int_if -> <network>) while the rule is read in the opposite direction (from <network> to <data>). Can you put some light on that? What rule should I use if I want to limit upload speed?

Last edited by ivanatora; 3rd December 2008 at 12:15 PM.
Reply With Quote
Old 4th December 2008
s2scott's Avatar
s2scott s2scott is offline
Package Pilot
 
Join Date: May 2008
Location: Toronto, Ontario Canada
Posts: 198
Thanked 9 Times in 8 Posts
Default

Quote:
Originally Posted by ivanatora View Post
I'm still wandering why (sic) queuing works for downloads (direction: <data> -> $ext_if -> $int_if -> <network>) while the rule is read in the opposite direction (from <network> to <data>). Can you put some light on that? What rule should I use if I want to limit upload speed?
Because...
Code:
pass in quick on $int_if \
 from <network> to <data> tag DO_NAT \
 keep state queue \
 data label "do_nat_data"
"keep state" sets up the return path INCLUDING, in your case, return path through your "queue." Your outbound packets establish state entry and the matching reply packets come through the state table entry, with your queue attached, and not the rules table.

/S
__________________
Never argue with an idiot. They will bring you down to their level and beat you with experience.

Last edited by s2scott; 4th December 2008 at 01:17 AM.
Reply With Quote
Old 9th December 2008
ivanatora ivanatora is offline
Real Name: Ivan
Fdisk Soldier
 
Join Date: Jul 2008
Location: Bulgaria
Posts: 51
Thanked 2 Times in 1 Post
Default

I'm here again with a bit different approach.
I've read that there is a PF option - "state-policy" that should be "if-bound" if I want to match traffic in different directions. Otherwise the created state would float from one interface to other and I could not shape upload and download differently.
I make one default queue for each interface, and two specialized 'limited' queues, too. I've wrote some rules, but the problem is that traffic never goes assigned to the limited queues. And it is pretty puzzling to me, because I see there are packets matching these labels.
Here again is my pf.conf:
Code:
### Macros
int_if = "re0"
ext_if = "rl0"
ext_ip = "192.168.1.2"

### Tables
table <network>  { 192.168.0.34, 192.168.0.223 }
table <tusite>  { 222.111.111.222 }
### Options
set state-policy if-bound
### Normalizations
scrub in all

### Queueing

altq on $int_if hfsc bandwidth 8Mb queue {download, gen_down}
queue gen_down bandwidth 4Mb hfsc (realtime 2Mb upperlimit 2Mb default)
queue download bandwidth 2Mb hfsc (realtime 1Mb upperlimit 1536Kb)

altq on $ext_if hfsc bandwidth 8Mb queue {upload, gen_up}
queue gen_up bandwidth 4Mb hfsc (realtime 2Mb upperlimit 2Mb default)
queue upload bandwidth 1Mb hfsc (realtime 512Kb upperlimit 768Kb)

### Translation
nat on $ext_if from <network> to any -> $ext_ip

### Filtering
pass in log (all to pflog0) quick on $int_if from <network> to <tusite> keep state tag to_data label "to_data" queue upload
pass out log (all to pflog1) quick on $ext_if tagged to_data label "from_data" queue download
Here is some output from pftop, showing all traffic goes assigned to the default queues:
Code:
pfTop: Up Queue 1-6/6, View: queue, Cache: 10000                           20:48:19

QUEUE               BW SCH  PR   PKTS  BYTES  DROP_P DROP_B QLEN BORR SUSP P/S  B/S
root_re0         8000K hfsc  0      0      0       0      0    0             0    0
 gen_down        4000K hfsc     63051 10784K      38  34460   42           226 248K
 download        2000K hfsc         0      0       0      0    0             0    0
root_rl0         8000K hfsc  0      0      0       0      0    0             0    0
 gen_up          4000K hfsc    112576   133M       0      0    0           225 171K
 upload          1000K hfsc         0      0       0      0    0             0    0
And here is the most annoying thing - seeing traffic per label in PF - I see there are packets matching these rules:
Code:
# pfctl -s label
to_data 205761 9262 8544433 3647 155866 5615 8388567
from_data 205755 9262 8544433 5615 8388567 3647 155866
If so, why there is no traffic in the specialized queues?
Reply With Quote
Old 9th December 2008
s2scott's Avatar
s2scott s2scott is offline
Package Pilot
 
Join Date: May 2008
Location: Toronto, Ontario Canada
Posts: 198
Thanked 9 Times in 8 Posts
Default

You cannot DIRECTLY rate-limit the traffic you RECEIVE traffic from your ISP. You CAN rate-limit the traffic you SEND (TX out an interface you/pf control).

As such, I think your queues are backwards.

QUEUE assignment(s) may be made (configured) on both in -and- out bound directions/interfaces; however, the queue-affect is in the OUT direction only of a configured interface.

/S
__________________
Never argue with an idiot. They will bring you down to their level and beat you with experience.

Last edited by s2scott; 9th December 2008 at 11:59 PM.
Reply With Quote
Old 9th December 2008
s2scott's Avatar
s2scott s2scott is offline
Package Pilot
 
Join Date: May 2008
Location: Toronto, Ontario Canada
Posts: 198
Thanked 9 Times in 8 Posts
Default

What is (are) the essential purpose(s)/goal(s) of your queue implementation?

/S
__________________
Never argue with an idiot. They will bring you down to their level and beat you with experience.
Reply With Quote
Old 11th December 2008
ivanatora ivanatora is offline
Real Name: Ivan
Fdisk Soldier
 
Join Date: Jul 2008
Location: Bulgaria
Posts: 51
Thanked 2 Times in 1 Post
Default

The purpose of that queueing is that I have a content web server that is reached via another link at my ISP. The situation is as follows:
1) Our network --> the GW I'm working on
2) the GW I'm working on --> our ISP
3.1) our ISP --> Internet
3.2) our ISP ----> content server at different link

The link to that content server is at very limited bandwidth, that's why I want to make sure noone could 'takeover' all the bandwidth.
So you say that queue download and queue upload should be swapped here?
Code:
pass in log (all to pflog0) quick on $int_if from <network> to <tusite> keep state tag to_data label "to_data" queue upload
pass out log (all to pflog1) quick on $ext_if tagged to_data label "from_data" queue download
You are right, I will check that a bit later.

Last edited by ivanatora; 11th December 2008 at 09:35 AM.
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Working with CVS? Zmyrgel OpenBSD General 15 6th October 2009 01:32 PM
[ OpenBSD 4.5 ] apm -C not working wraith0x2b OpenBSD Installation and Upgrading 17 6th May 2009 09:03 AM
USB not working after suspend stukov Other BSD and UNIX/UNIX-like 5 11th August 2008 06:48 PM
pf.conf brute force rule ijk FreeBSD Security 6 11th August 2008 04:54 PM
Crontab not working beandip FreeBSD General 6 6th August 2008 08:33 PM


All times are GMT. The time now is 05:48 AM.


Powered by vBulletin® Version 3.8.4
Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content copyright © 2007-2010, the authors
Daemon image copyright ©1988, Marshall Kirk McKusick