|
OpenBSD General Other questions regarding OpenBSD which do not fit in any of the categories below. |
|
Thread Tools | Display Modes |
|
|
|||
Multi-Path or Route-To?
Hello everyone,
I'd like to ask all of those who have serious in-depth knowledge of how the OpenBSD kernel works and how pf works to answer a possibly simple, possibly complicated question. Basically what I want to know is how the multipath routing in the OpenBSD kernel works. Is it meant to work for multiple routers available on the same subnet or can they be on separate subnets? I am trying to use only 1 NIC for egress traffic to the Internet. Both routes are actually 2 separate ISP connections (say cable and dsl) over a single NIC with both lines connected to it via a switch and ISP1's IP assigned to the NIC as the primary IP and the IP of ISP2 aliased onto the same NIC and then using multipath default routes to define the default gateways of each ISP connection. What I'm basically trying to do it load balance 2 Internet connections with my OBSD firewall. But with simple multipath routes it sort of works, but I suffer a whole heck of a lot of packet loss. What I ultimately found using line snooping Ethernet sniffers is that the firewall would sometimes attempt to send traffic from one ISP's IP address to the default gateway of the other ISP's connection. I assumed this would not happen since each ISP connection has a small 5 IP subnet and I did specify the netmask and the default gateway for each ISP is within its subnet. I made the seemingly logical assumption that it would intelligently route traffic from ISP1's IP to ISP1's default gateway and ISP2's IP to ISP2's default gateway. But it doesn't seem to be doing that, picking a default gateway seemingly at random without taking into consideration the source IP its employing at the moment. Why is this? Am I misunderstanding what multipath routes are meant for? Or is this a bug in the kernel/my configuration of multipath? I found a seemingly slightly better working solution with just using Route-to rules in pf. I now no longer suffer from packets going down the wrong default gateway, but I only have the default gateway for ISP1's connection defined for this to work right. Is multipath routing only meant to work for routers on the same subnet? |
|
|||
> Alias IP addresses are not designed for routing. They merely give the NIC multiple IP > addresses to respond to. But they will always respond with their "real" address as > the origination.
Thanks, I was not aware of this. Would explain the still somewhat odd behavior I am seeing with UDP traceroutes while other things TCP based (ie web surfing) seems to be working flawlessly. |
|
|||
Now with 2 NICs installed...
Now I'd like to understand with 2 NICs installed for egress traffic, 1 for each ISP connection. With multipath default routes to each ISP's respective default gateway next hops, why is it that the box only uses the secondary ISPs connection exclusively?
If I issue a $ping 8.8.8.8 The ping times are appropriate for the usual latency of the secondary connection. This is confirmed by issuing a $traceroute 8.8.8.8 and watching traffic go down that route. If I issue a $ping -I {IP of the primary connection} 8.8.8.8 The pings work, and have the usual latency that can be expected on that line. This can also be confirmed by issuing a $traceroute -s {IP of the primary connection} 8.8.8.8 and watching the traffic go down the primary line this time. Then to really make matters worse, if I issue a $ping -I {IP of the secondary connection} 8.8.8.8 This fails....as does $traceroute -s {IP of secondary connection} 8.8.8.8 Why? It should work if it works when not specifying any address. And when not specifying any address, shouldn't it pick a connection to use at random or using a round-robin method with multipath routing enabled, or am I misunderstanding something again? |
|
||||
It appears you did not follow FAQ 6.14, or did not follow it correctly. Note the section on sysctl settings.
From route(8), highlight mine: Quote:
|
|
|||
Right, I did this...hence my confusion...I did read the that. What it conveniently leaves out is if this is meant to have multiple routes defined for a single source IP, or does it still work if you have more than one IP with default gateways each to the same basic (Internet) destination.
|
|
|||
NIC1 has an IP from the subnet of the primary ISP. And NIC2 has an IP from the secondary ISP. They are on totally separate subnets from two totally different ISPs.
|
|
||||
OK. There are a few possibilities for misconfiguration, such as a manual sysctl(8) command but a forgotten edit of sysctl.conf(5) and then a reboot. But just to be sure, post the outputs of:
|
|
|||
Quote:
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 33200 priority: 0 groups: lo inet 127.0.0.1 netmask 0xff000000 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5 re0: flags=8b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,S IMPLEX,MULTICAST> mtu 1500 lladdr 00:14:d1:1c:ee:a8 priority: 0 media: Ethernet autoselect (100baseTX full-duplex) status: active inet6 fe80::214:d1ff:fe1c:eea8%re0 prefixlen 64 scopeid 0x1 re1: flags=8b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,S IMPLEX,MULTICAST> mtu 1500 lladdr 00:14:d1:1c:f3:f7 priority: 0 media: Ethernet autoselect (1000baseT full-duplex,master) status: active inet 192.168.xxx.yyy netmask 0xfffffe00 broadcast 192.168.xxx.255 inet6 fe80::214:d1ff:fe1c:f3f7%re1 prefixlen 64 scopeid 0x2 msk0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MU LTICAST> mtu 1500 lladdr 00:1b:b9:b0:07:80 priority: 0 media: Ethernet autoselect (100baseTX full-duplex) status: active inet6 fe80::21b:b9ff:feb0:780%msk0 prefixlen 64 scopeid 0x3 enc0: flags=0<> priority: 0 groups: enc status: active pfsync0: flags=41<UP,RUNNING> mtu 1500 priority: 0 pfsync: syncdev: re1 maxupd: 128 defer: off groups: carp pfsync pflog0: flags=141<UP,RUNNING,PROMISC> mtu 33200 priority: 0 groups: pflog carp0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500 lladdr 00:00:5e:00:01:02 priority: 0 carp: MASTER carpdev re0 vhid 2 advbase 1 advskew 0 groups: carp egress status: master inet6 fe80::200:5eff:fe00:102%carp0 prefixlen 64 scopeid 0x6 inet 97.67.xxx.yyy netmask 0xfffffff8 broadcast 97.67.xxx.yyy carp1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500 lladdr 00:00:5e:00:01:01 priority: 0 carp: MASTER carpdev re1 vhid 1 advbase 1 advskew 0 groups: carp status: master inet6 fe80::200:5eff:fe00:101%carp1 prefixlen 64 scopeid 0x7 inet 192.168.xxx.yyy netmask 0xfffffe00 broadcast 192.168.xxx.255 carp2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500 lladdr 00:00:5e:00:01:03 priority: 0 carp: MASTER carpdev msk0 vhid 3 advbase 1 advskew 0 groups: carp egress status: master inet6 fe80::200:5eff:fe00:103%carp2 prefixlen 64 scopeid 0x8 inet 68.213.xxx.yyy netmask 0xfffffff8 broadcast 68.213.xxx.yyy Here is $route -n show -inet Routing tables Internet: Destination Gateway Flags Refs Use Mtu Prio Iface default 68.213.xxx.yyy UGSP 0 17 - 8 carp2 default 97.67.xxx.yyy UGSP 0 9 - 8 carp0 68.213.xxx.yyy/29 link#8 UC 1 0 - 4 carp2 68.213.xxx.yyy 00:22:10:a3:22:e0 UHLc 1 0 - 4 carp2 97.67.xxx.yyy/29 link#6 UC 1 0 - 4 carp0 97.67.xxx.yyy 00:a0:c8:5b:1c:bb UHLc 1 0 - 4 carp0 127/8 127.0.0.1 UGRS 0 0 33200 8 lo0 127.0.0.1 127.0.0.1 UH 1 60 33200 4 lo0 192.168.xxx/23 link#2 UC 9 0 - 4 re1 192.168.xxx.yyy 192.168.xxx.yyy UH 0 0 - 4 carp1 224/4 127.0.0.1 URS 0 0 33200 8 lo0 Here are the /etc/hostnames.* /etc/hostname.carp0 inet 97.67.xxx.yyy 255.255.255.248 97.67.xxx.yyy vhid 2 pass password1 carpdev re0 !route add -mpath default 97.67.37.89 /etc/hostname.carp1 inet 192.168.xxx.1 255.255.254.0 192.168.xxx.255 vhid 1 pass password2 carpdev re1 /etc/hostname.carp2 inet 68.213.xxx.yyy 255.255.255.248 68.213.xxx.yyy vhid 3 pass password3 carpdev msk0 !route add -mpath default 68.213.215.9 /etc/hostname.msk0 up /etc/hostname.pfsync0 up syncdev re1 /etc/hostname.re0 up /etc/hostname.re1 inet 192.168.xxx.11 255.255.254.0 192.168.xxx.255 up rtsol I changed the IP addresses for the usual privacy reasons. |
|
||||
I believe your carp(4) implementation might be getting in the way. As configured, carp is not being utilized for load balancing.. You might consider using real interfaces. See if it makes a difference.
If not, or for some real help instead of my fumblings, consider posting to misc@. Next time, please enclose such pasted content in code tags. It makes things like routing tables much easier to read. As posted, I'm having trouble with it. |
|
|||
Sorry about not using the code tags...not familiar with that. Will have to spend some time on the forums guides to learn how to do this.
As for the carp interfaces, I'm using them between to similarly configured boxes to not only load balance 2 Internet connections, but also to be redundant firewalls of each other. So technically they are being used. If they are working the way I expect...that's another thing. What makes you say they aren't being used, as is currently configured? |
|
|||
I'm not entirely sure what you mean by that. If I understand correctly, I'm trying to achieve IP load balancing...
|
|
|||
This forum uses BBcode, which is similar to HTML markup.. simply place the text between the tags.
[code]Formatted output goes here.[/code] |
|
|||
Oh I apologize...I misunderstood. I am not trying to use carp for any load balancing. Just for fail-over...should one firewall die/get disconnected/administratively shutdown for whatever reason...the other takes over. This does work as expected..am quite happy with this.
I'm attempting to use Equal Cost Multipath routing to load balance traffic from the Internal LAN (and the firewall itself I would assume) between the two Internet connections. This is what is not working. |
|
|||
Should I just do away with multipath routes and just use pf route-to rules for this?
|
Thread Tools | |
Display Modes | |
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
How to add static route using virtual NIC | bsdplus | Solaris | 1 | 22nd August 2010 02:10 AM |
add to path | carpman | FreeBSD General | 4 | 19th August 2010 06:13 AM |
ping: sendto: No route to host | joostvgh | OpenBSD General | 2 | 29th April 2010 12:34 PM |