View Single Post
  #1   (View Single Post)  
Old 12th January 2014
Atlantis Atlantis is offline
New User
 
Join Date: Jan 2014
Posts: 4
Default OpenBSD-5.4::MPATH MULTIHOMED PF config issue

greetings All !

I encounter problems while setting up a multihomed FW/GW under OpenBSD 5.4
To make things easier, i enabled multipath into sysctl.conf (tried both equal and unequal).
Setup is as follow


Code:
INTERNET ===|(dsl:82.22X.XX.XX) FREE-DSL-BOX (int:192.168.1.254) |===| (rl0)             (em1) |=== LAN
								|	     OPENBSD BOX       | 
INTERNET ===|(fttb:81.xx.xx.xx) NC-FTTB-BOX (int:192.168.0.1) |======| (re0)             (em0) |=== DMZ
=======

Now the question

Setup is working, but not so good. I would need to have all traffic to pass through NC in priority, and , in case of failure, switch to FREE (NC connection bandwidth is 200MBps, Free is 14MBps).
In parallel, i would need the GW to be reachable from FREE as well as NC networks (both providers boxes do NAT traffic as well...)
I am able to proceed when i disable either re0 or rl0, but when both are enabled, only one is working...
I believe it has something to do with reply-to sent to the wrong interface (?)
In parallel, i have a lot of packets losses, for an unknown reason (i log everything for now - debugging purposes)
Any ideas about how to setup this conf correctly and/or to optimize this setup ?
BTW: i tried to setup -mpath using both equal and unequal weights, without success, pb remains the same...

Thanks for your help !

sincerely
___________________
here are the interfaces and setup description:

===
Code:
# ifconfig

lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 33192
        priority: 0
        groups: lo
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x6
        inet 127.0.0.1 netmask 0xff000000
rl0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
        lladdr XX:XX:18:XX:XX:7d
        description: Connexion Free
        priority: 0
        groups: egress
        media: Ethernet autoselect (100baseTX full-duplex)
        status: active
        inet 192.168.1.253 netmask 0xffffff00 broadcast 192.168.1.255
        inet6 fe80::2e0:18ff:XXXX:XXXX%rl0 prefixlen 64 scopeid 0x1
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
        lladdr XX:XX:6e:XX:XX:XX
        priority: 0
        media: Ethernet autoselect (1000baseT full-duplex,master)
        status: active
        inet 10.0.0.254 netmask 0xffffff00 broadcast 10.0.0.255
        inet6 fe80::XXX:6eff:XXXX:XXXX%em0 prefixlen 64 scopeid 0x2
em1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
        lladdr 00:30:6e:XX:XX:XX
        priority: 0
        media: Ethernet autoselect (1000baseT full-duplex)
        status: active
        inet 172.16.1.254 netmask 0xffffff00 broadcast 172.16.1.255
        inet6 fe80::230:XXXX:XXXX:8e81%em1 prefixlen 64 scopeid 0x3
re0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
        lladdr 00:1e:XX:XX:XX:XX
        description: Connexion Numericable
        priority: 0
        groups: egress
        media: Ethernet autoselect (1000baseT full-duplex,rxpause,txpause)
        status: active
        inet 192.168.0.253 netmask 0xffffff00 broadcast 192.168.0.255
        inet6 feXX::21e:XXXX:feXX:83XX%re0 prefixlen 64 scopeid 0x4
enc0: flags=0<>
        priority: 0
        groups: enc
        status: active
pflog0: flags=141<UP,RUNNING,PROMISC> mtu 33192
        priority: 0
        groups: pflog
=======



Routing table is:
====================

Code:
# route show
Routing tables

Internet:
Destination        Gateway            Flags   Refs      Use   Mtu  Prio Iface
default            192.168.0.1        UGSP       4   345002     -     8 re0
default            192.168.1.254      UGSP       4   444011     -     8 rl0
10.0.0/24          link#2             UC         3        0     -     4 em0
10.0.0.150         00:0c:XX:22:XX:f5  UHLc       1    40586     -     4 em0
loopback           localhost          UGRS       0        0 33192     8 lo0
localhost          localhost          UH         2     2154 33192     4 lo0
172.16.1/24        link#3             UC         3        0     -     4 em1
192.168.1.253      00:XX:XX:2a:XX:7d  UHLc       0      212     -     4 lo0
192.168.0/24       link#4             UC         2        0     -     4 re0
192.168.0.1        24:ec:XX:05:XX:4X  UHLc       1        5     -     4 re0
base-address.mcast localhost          URS        0        0 33192     8 lo0
===================

DMZ contains a mail server, which grabs mails from public services (yahoo,...) using pop3s, ntp server, and dns server.
Gateway itself manages with multidomain smtp routing, and incoming mails are forwarded to internal mail server, as well as DNS
===================

Here is the PF.conf setup

================

Code:
############################ INTERFACES INTERNES #################################

orange_if  = "em0"
orange_net = "10.0.0.0/24"

green_if  = "em1"
green_net = "172.16.1.0/24"

############################ INTERFACES EXTERNES #################################

nc  = "re0"
free = "rl0"

############################ PASSERELLES EXTERNES #################################

nc_gw = "192.168.0.1"
free_gw = "192.168.1.254"


############################ TABLES ##################################

table <ournets> persist { 10.0.0.0/24, 172.16.1.0/24 }
table <bruteforce> persist
table <ossec_fwtable> persist # ossec_fwtable
table <allowed_out> persist { }
table <firewall> const { self }

############################ PARAMETRES ##############################

set state-policy floating
set block-policy drop
set optimization normal
#set require-order yes
set timeout { interval 10, frag 30 }
set timeout { tcp.first 120, tcp.opening 30, tcp.established 86400 }
set timeout { tcp.closing 900, tcp.finwait 45, tcp.closed 90 }
set timeout { udp.first 60, udp.single 30, udp.multiple 60 }
set timeout { icmp.first 20, icmp.error 10 }
set timeout { other.first 60, other.single 30, other.multiple 60 }
set timeout { adaptive.start 0, adaptive.end 0 }
set limit { states 5000, frags 2000 }
set limit table-entries 500000
set fingerprints "/etc/pf.os"
set skip on lo0

########################## fragment reassemble  ############################
# Normalisation de tous les paquets entrants.
match in all scrub (no-df)
#scrub out all fragment reassemble max-mss 1400

###################################### N  A  T  ##########################################

#  nat outgoing connections on each internet interface
match out on egress inet from !(egress:network) to any nat-to { egress }

###################################### REGLES DENY  #########################################

# block unwanted hosts
block in quick from <bruteforce>
block in quick from <ossec_fwtable>

# block anything by default
block in log
block out log

# ORANGE TO GREEN NOK
block drop in on $green_if from $orange_net to $green_net

###################################### REGLES ALLOW #########################################

# Allow ICMP on external interfaces

pass in quick on $green_if proto icmp from {$green_net, $orange_net} to any nat-to egress keep state


#pass in inet proto tcp from {$green_net, $orange_net} to any port { http, https } divert-to 127.0.0.1 port 3128

#  pass all outgoing packets on internal interface

pass out quick log on $green_if to $green_net
pass out quick log on $orange_if to $orange_net

################################################################################

# Autoriser le trafic sortant et entrant sur le r?seau local.
# ces r?gles cr?eront des entr?es au niveau de la table d'?tat ?tant
# donn? que le mot-cl? "keep state" est automatiquement appliqu?.

# on autorise le LAN et certains services a atteindre le firewall
pass in quick log on $green_if proto tcp from $green_net to ($green_if) port { ssh, 3128 } keep state
pass in quick log on $green_if proto udp from $green_net to ($green_if) port 53

# on autorise certains services en zone DMZ a etre atteints par le LAN

pass in quick log on $green_if proto tcp from $green_net to $orange_net port { smtp, http, https, ssh, 137, 139, 445, 993 } keep state
pass in quick log on $orange_if proto udp from $orange_net to !$green_net port { domain }
pass in quick log on $orange_if proto tcp from $orange_net to !$green_net port {25, 110, 993, 995 }

# on autorise la sortie sur internet

# Allow ICMP on external interfaces
pass in quick on $free proto icmp from <firewall> to any keep state
pass in quick on $nc proto icmp from <firewall> to any keep state


pass in quick log on $green_if proto tcp from $green_net to !$orange_net port { http, https }


pass out log on egress proto udp from { <firewall>, $green_net, $orange_net } to any port { domain, ntp } keep state (if-bound)
pass out log on egress proto tcp from { <firewall>, $green_net } to any port { domain, http, https, ntp } keep state (if-bound)
pass out log on egress proto tcp from $orange_net to any port { domain, smtp, ntp } keep state (if-bound)
pass out log on egress inet proto icmp all icmp-type { echoreq }

########################### REGLES EN ENTREE ##############################
######################## SERVICES EN ENTREE #########################

###################### E/SMTP/IMAPS MOBILE VIA FREE ######################
pass in quick log on $free inet proto tcp from any to $free port = 25 reply-to ($free $free_gw) flags S/SA keep state
#pass in quick log on $free inet proto tcp from any to $free port = 9930 reply-to ($free $free_gw) flags S/SA keep state (if-bound) rdr-to 10.0.0.150 port 993

pass in quick log on $free inet proto tcp from XX.84.144.0/22 to $free port = 9930 reply-to ($free $free_gw) flags S/SA keep state (if-bound) rdr-to 10.0.0.150 port 993
pass in quick log on $free inet proto tcp from XX.84.144.0/22 to $free port = 9931 reply-to ($free $free_gw) flags S/SA keep state (if-bound) rdr-to 10.0.0.150 port 25

pass in quick log on $free inet proto tcp from XX.10.159.0/24 to $free port = 9930 reply-to ($free $free_gw) flags S/SA keep state (if-bound) rdr-to 10.0.0.150 port 993
pass in quick log on $free inet proto tcp from XX.10.159.0/24 to $free port = 9931 reply-to ($free $free_gw) flags S/SA keep state (if-bound) rdr-to 10.0.0.150 port 25

pass in quick log on $free inet proto tcp from XX.160.0.0/12  to $free port = 9930 reply-to ($free $free_gw) flags S/SA keep state (if-bound) rdr-to 10.0.0.150 port 993
pass in quick log on $free inet proto tcp from  XX.8.160.0/12 to $free port = 9931 reply-to ($free $free_gw) flags S/SA keep state (if-bound) rdr-to 10.0.0.150 port 25
pass in quick log on $free inet proto tcp from any to $free port = 25 reply-to ($free $free_gw) flags S/SA keep state

################### E/SMTP/IMAPS MOBILE VIA NUMERICABLE ##################
#pass in quick log on $nc inet proto tcp from any to $nc port = 9930 reply-to ($nc $nc_gw) flags S/SA keep state (if-bound) rdr-to 10.0.0.150 port 993

pass in quick log on $nc inet proto tcp from XX.84.144.0/22 to $nc port = 9930 reply-to ($nc $nc_gw) flags S/SA keep state (if-bound) rdr-to 10.0.0.150 port 993
pass in quick log on $nc inet proto tcp from XX.84.144.0/22 to $nc port = 9931 reply-to ($nc $nc_gw) flags S/SA keep state (if-bound) rdr-to 10.0.0.150 port 25

pass in quick log on $nc inet proto tcp from XX.10.159.0/24 to $nc port = 9930 reply-to ($nc $nc_gw) flags S/SA keep state (if-bound) rdr-to 10.0.0.150 port 993
pass in quick log on $nc inet proto tcp from XX.10.159.0/24 to $nc port = 9931 reply-to ($nc $nc_gw) flags S/SA keep state (if-bound) rdr-to 10.0.0.150 port 25

pass in quick log on $nc inet proto tcp from XX.160.0.0/12  to $nc port = 9930 reply-to ($nc $nc_gw) flags S/SA keep state (if-bound) rdr-to 10.0.0.150 port 993
pass in quick log on $nc inet proto tcp from XX.160.0.0/12  to $nc port = 9931 reply-to ($nc $nc_gw) flags S/SA keep state (if-bound) rdr-to 10.0.0.150 port 25
===================

Last edited by Atlantis; 13th January 2014 at 05:21 PM.
Reply With Quote