DaemonForums  

Go Back   DaemonForums > OpenBSD > OpenBSD General

OpenBSD General Other questions regarding OpenBSD which do not fit in any of the categories below.

Reply
 
Thread Tools Display Modes
  #1   (View Single Post)  
Old 3rd March 2017
leos leos is offline
Port Guard
 
Join Date: Mar 2017
Posts: 11
Default How to shape traffic using PF?

hello guys
I have a very strange problem trying to set up a traffic shape
I followed all the documentation and I built this PF file:

ext_if="sis0"
int_if="sis1"

table <usr1_direct> { 192.168.1.210, 192.168.1.211 }

int_ip="192.168.1.228"
int_lan="192.168.1.0/24"
ext_ip="X.Y.Z.A"

up_max="1Mb"
up_slow="100Kb"
up_fast="500Kb"

dn_max="1Mb"
dn_slow="100Kb"
dn_fast="500Kb"

set block-policy drop
set state-policy if-bound
set loginterface $ext_if
set skip on lo

queue up_parent on $ext_if bandwidth $up_max
queue up_default parent up_parent bandwidth $up_slow default
queue up_usr_fast parent up_parent bandwidth $up_fast


queue dn_parent on $int_if bandwidth $dn_max
queue dn_default parent dn_parent bandwidth $dn_slow default
queue dn_usr_fast parent dn_parent bandwidth $dn_fast


match out on $ext_if from $int_if:network to any nat-to ($ext_if)
match on $ext_if scrub (no-df random-id max-mss 1440)

block log all

pass in quick on $ext_if inet proto icmp to ($ext_if) icmp-type echoreq


pass out on $ext_if inet queue up_default
pass out on $ext_if inet from <usr1_direct> queue up_usr_fast

pass in on $int_if proto udp from $int_lan to any port 53

pass in on $int_if proto icmp from $int_lan to any
pass in on $int_if proto { tcp, udp } from $int_lan to any queue up_usr_fast

pass out on $int_if inet queue dn_default no state
pass out on $int_if inet to <usr1_direct> queue dn_usr_fast no state



querying the systat queue I can see:

QUEUE BW SCH PRIO PKTS BYTES DROP_P DROP_B QLEN BORROW SUSPEN P/S B/S
up_parent on sis0 1M 0 0 0 0 0
up_default 100K 19506 18184098 0 0 0
up_usr_fast 500K 0 0 0 0 0
dn_parent on sis1 1M 0 0 0 0 0
dn_default 100K 20093 18309620 0 0 0
dn_usr_fast 500K 166 26306 0 0 0


and I assume it's OK but... trying with a speedtest I can see I'm using the full bandwidth (8Mbit) and not the assigned.. I'm sure I'm wrong somewhere but it's days I going mad... anyone can help me please?
Thanks a lot, Leo
Reply With Quote
  #2   (View Single Post)  
Old 3rd March 2017
jggimi's Avatar
jggimi jggimi is online now
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 6,383
Default

Hello, and welcome!

You are using the full bandwidth because the values you have set are target bandwidths to use when contention occurs. You have called your variables "max" but using them as targets in your rules. For example, your rule:
Code:
queue up_parent on $ext_if bandwidth $up_max
is interpreted as:
Code:
queue up_parent on sis0 bandwidth 1M
You should be able to see this with # pfctl -sq. If you want to set maximums, you'll need to do so according to the syntax, assigning a target and then adding "max <value><suffix>" to the rule. See the QUEUEING section of the pf.conf(5) man page.

I'm going to guess that traffic is assigned to the default queues because the assignment of queue occurs when the state is first established. (With keep state, once a state is established, PF rules are no longer inspected.) Your traffic is "default" because you're using different queues on each interface, and only the queue defined at state creation is associated with the state. That same section of the pf.conf(5) man page says that the default queue will be used whenever there is no match.

I hope this helps.
Reply With Quote
  #3   (View Single Post)  
Old 6th March 2017
leos leos is offline
Port Guard
 
Join Date: Mar 2017
Posts: 11
Default

Quote:
Originally Posted by jggimi View Post
Hello, and welcome!
Thanks and thanks for reply!

Quote:
Originally Posted by jggimi View Post
Code:
queue up_parent on $ext_if bandwidth $up_max
is interpreted as:
Code:
queue up_parent on sis0 bandwidth 1M
infact I did it, and I would expect to see 1Mb up with a speedtest but, whatever I set, I always get 8Mbit (the max bandwidth of the connection I'm using for tests)
or, more probably, I misunderstand you...
Reply With Quote
  #4   (View Single Post)  
Old 6th March 2017
jggimi's Avatar
jggimi jggimi is online now
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 6,383
Default

It is not clear to me what you did. According to pf.conf(5), the syntax is:
Code:
queue my_child parent my_parent bandwidth 1M max 2M
Are you using this syntax?

I have never provisioned maximums or minimums, as I have no use-case for them.

Last edited by jggimi; 6th March 2017 at 11:51 AM. Reason: clarity
Reply With Quote
  #5   (View Single Post)  
Old 6th March 2017
leos leos is offline
Port Guard
 
Join Date: Mar 2017
Posts: 11
Default

NOOO I can't beleive!!
I didn't set it correctly, I miss "max $MAX" and it didn't work... now I set it and it works perfectly!
thanks a lot for this tip!!!
Reply With Quote
  #6   (View Single Post)  
Old 6th March 2017
jggimi's Avatar
jggimi jggimi is online now
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 6,383
Default



I'm very glad you got it working.
Reply With Quote
  #7   (View Single Post)  
Old 7th March 2017
leos leos is offline
Port Guard
 
Join Date: Mar 2017
Posts: 11
Default

Hello again, I'm still having problems because it works great for "upstream" (I can define whatever I want and it's ok), the problem I'm experiencing is in "downstream". I can define anything but it always use queue "default"

Setting a pass rule and debugging it with "log":

pass in log on $ext_if from any to <slow> queue dn_slow
pass out log on $int_if from any to <slow> queue dn_slow

the rule never matches and it makes sense because there's a NAT and (I think) a stateful in rule "pass in on $int_if blablabla".
I tried to put a "no state" but there's no way to let the packet go back to the internal IP

will be anyone so kind to help me again?
Reply With Quote
  #8   (View Single Post)  
Old 7th March 2017
jggimi's Avatar
jggimi jggimi is online now
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 6,383
Default

Stateful processing is the default, and recommended, because without it every packet is inspected for rule matching by the kernel.

Instead of having "upload" and "download" queues, consider using the same queue name on both your interfaces. Each interface may have its own bandwidth shaping provisions, but will use the queue name established by whichever matching rule applies when the state is first established.
Reply With Quote
  #9   (View Single Post)  
Old 7th March 2017
jggimi's Avatar
jggimi jggimi is online now
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 6,383
Default

Here's the example from pf.conf(5), for interface em0:

Code:
queue std on em0 bandwidth 100M 
queue ssh parent std bandwidth 10M 
queue mail parent std bandwidth 10M 
queue http parent std bandwidth 80M default
Let us pretend em0 is an external interface, and we have a Gigabit local network on em1. I could use these same queue names, with different shaping values, by adding this set of lines below. Note that each queue has 10x the bandwidth assigned on this 10x faster internal network, and the names are exactly the same:
Code:
queue std on em1 bandwidth 1000M 
queue ssh parent std bandwidth 100M 
queue mail parent std bandwidth 100M 
queue http parent std bandwidth 800M default

Last edited by jggimi; 7th March 2017 at 05:39 PM. Reason: typo - forgot (5) on pf.conf
Reply With Quote
Old 8th March 2017
leos leos is offline
Port Guard
 
Join Date: Mar 2017
Posts: 11
Default

uhm, in my case doesn't work... it say (in case of http) queue http respecified...
maybe I didn't explained well what I'm trying to do... I have a network with "standard users" and "privileged users" (I'd like to assign for example synchronous 1Mb to standard users and 2Mb to privileged users), both of them are on the same subnet (192.168.1.0/24)
there's a guest subnet where are connected guests, this is connected to another interface and obviously another subnet (192.168.2.0/24), and I'll give them as minimum as possible to let them navigate the Internet.

the bandwidth of this Internet connection is a synchronous 8Mb...

I'm going to define tables with different kind of users (at the moment I defined just one table, if I'm not able to set properly the shaping, I'm stuck)..

this is my partial working PF:


int_if="sis0"
ext_if="sis1"

table <slow> { 192.168.1.210 }

int_lan="192.168.1.0/24"

set block-policy drop
set state-policy if-bound
#set loginterface $ext_if
set skip on lo

### QUEUE
queue up_parent on $ext_if bandwidth 7M
queue up_default parent up_parent bandwidth 512K max 1M default
queue up_slow parent up_parent bandwidth 128K max 256K

queue dn_parent on $int_if bandwidth 7M
queue dn_default parent dn_parent bandwidth 512K max 1M default
queue dn_slow parent dn_parent bandwidth 128K max 256K

### MATCH RULES
match out on $ext_if from $int_if:network to any nat-to ($ext_if)
match on $ext_if scrub (no-df random-id max-mss 1440)

block log all

pass out on $ext_if

#pass in quick on $ext_if inet proto icmp to ($ext_if) icmp-type echoreq

pass in on $int_if proto icmp from $int_lan

pass in on $int_if proto { tcp, udp } from <slow> to any queue up_slow
pass out on $ext_if inet from <slow> queue up_slow

#pass in on $ext_if to <slow> queue dn_slow
#pass out on $int_if to <slow> queue dn_slow


I say partial because "speedtest.net" show me an upstream of 0,25Mbit, but whatever I set, I always get a downstream rate equal to what I defined as "standard". (1Mb in this case, I tried to set 3 and 3 is...)
Is not a problem to set just one queue, but I cannot figure how to set the downstream rate
(now the rule for downstream is remarked because it didn't work in any way...) I really like to understand how to fix it... sorry for still bugging
Reply With Quote
Old 8th March 2017
jggimi's Avatar
jggimi jggimi is online now
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 6,383
Default

Here is an actual set of queue definitions from one of my firewalls.

The Internet connection is 10 megabits per second up, 75 megabits per second down. The ISP supports burst traffic, and the NICs on this firewall are all 100 megabit. Two of the NICs are trunked to provide 200Mbps bandwidth on the local gigabit network..

It's really very simple. When there is contention for resource, and queues occur, the heavy user (my son) gets 50% of the available bandwidth. When there is no contention, queues don't occur and therefore this won't matter.

I have highlighted the queues that share the same name:

Code:
queue up on $isp_nic bandwidth 10M burst 100M for 800ms
   queue son parent up bandwidth 5M
   queue std parent up bandwidth 5M default 

queue internal on $internal_nics bandwidth 200M
   queue local parent internal bandwidth 200M
   queue down parent internal bandwidth 75M burst 100M for 1600ms
      queue son parent down bandwidth 38M
      queue std parent down bandwidth 37M default
Reply With Quote
Old 9th March 2017
leos leos is offline
Port Guard
 
Join Date: Mar 2017
Posts: 11
Default

Ok it's official: I owe you a beer

Following your tips I was able to set two tables (fast and slow) that exactly respect my bandwidth limitation!
I also created a new rule for the "guests" interface adding a proper rule on "up" queue and creating a new rule for its interface to properly set the downstream.

As I can see it's enough to have just one "pass" rule and it matches both up and down bandwidth and this makes the pf file really simple and light:

pass in on $int_if proto { tcp, udp } from <slow> to !$dmz_lan queue slow

(Obviously I did the same for fast table and guests and whatever I set)

Thanks again Jggimi, your help is really appreciated, without your support I'm sure I'd never got it working!
Reply With Quote
Old 9th March 2017
jggimi's Avatar
jggimi jggimi is online now
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 6,383
Default

I'm very glad you got it working.

Really, the hardest part with this new queueing system is learning just how simple it can be.
Reply With Quote
Old 9th March 2017
leos leos is offline
Port Guard
 
Join Date: Mar 2017
Posts: 11
Default

Quote:
Originally Posted by jggimi View Post
I'm very glad you got it working.

Really, the hardest part with this new queueing system is learning just how simple it can be.
yes I agree... I'm trying to understand it deeply because at the end, the application is really simple, I'm still a little bit confused on the queue definition but having a working configuration it helps a lot

Really good thanks again!
Reply With Quote
Old 16th March 2017
leos leos is offline
Port Guard
 
Join Date: Mar 2017
Posts: 11
Default

uhm... I made all the things on the firewall online and doing some test I'm having a little doubt..
with nobody in the office the tested queues give me exactly the bandwidth I set (testing with speedtest.net)
now I have people in the office and.. testing from a PC I can see lower results... so, does PF take the amount I set and shares with all users automatically?
if PF does this by itself "automagically" it's really very useful!
Reply With Quote
Old 16th March 2017
jggimi's Avatar
jggimi jggimi is online now
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 6,383
Default

PF doesn't have the ability to match rules with users unless the packet being tested originates on the computer running PF.

PF tests packets -- and if queuing is used, assigns queues -- when packets arrive from any NIC that require evaluation, such as when establishing state.

PF only assigns queues based on your rules. Those rules include packets moving over multiple NICs, in multiple directions.

So, if it is working correctly when packets are placed on the appropriate queues, according to how you defined them, and if that is the way you intend them to work.
Reply With Quote
Old 17th March 2017
leos leos is offline
Port Guard
 
Join Date: Mar 2017
Posts: 11
Default

Ok so PF will share the bandwidth between all users:
if there is only one user he will have the max bandwidth, if there are 10 users they will share the same total bandwidth without saturating...
this is wonderful because this allow me to create the queues I need and I just have to made the sum of them lower than bandwidth available!
Reply With Quote
Old 17th March 2017
jggimi's Avatar
jggimi jggimi is online now
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 6,383
Default

Sorry for being a little extra pedantic, but this needs to be clear:

A "user" for PF is a specific filtering value, used on packets which originate or terminate on the system where PF is running. From pf.conf(5):
Code:
     user user
             This rule only applies to packets of sockets owned by the
             specified user.  For outgoing connections initiated from the
             firewall, this is the user that opened the connection.  For
             incoming connections to the firewall itself, this is the user
             that listens on the destination port.
Otherwise, PF merely manages network traffic, as rules define, without consideration of "users."
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Traffic Shaping Using PF Amithapr OpenBSD General 11 15th March 2017 05:07 PM
PF only firefox traffic ? bryn1u OpenBSD Security 12 7th November 2014 04:39 AM
Traffic between two vpn networks bertj FreeBSD Security 4 31st January 2013 02:44 PM
multiplexing traffic schmurfy OpenBSD General 6 26th March 2012 12:46 PM
PF Blocking VPN Traffic plexter OpenBSD Security 6 23rd January 2009 05:25 PM


All times are GMT. The time now is 04:38 PM.


Powered by vBulletin® Version 3.8.4
Copyright ©2000 - 2018, Jelsoft Enterprises Ltd.
Content copyright © 2007-2010, the authors
Daemon image copyright ©1988, Marshall Kirk McKusick