DaemonForums  

Go Back   DaemonForums > OpenBSD > OpenBSD General

OpenBSD General Other questions regarding OpenBSD which do not fit in any of the categories below.

Reply
 
Thread Tools Display Modes
  #1   (View Single Post)  
Old 16th January 2019
brudan brudan is offline
Fdisk Soldier
 
Join Date: Dec 2018
Posts: 74
Default how to mark packets for custom route?

My personal laptop runs OpenBSD. I browse the web using WiFi + openvpn client for added privacy. The default behavior of all outgoing traffic going through VPN is generally desirable, but I need some outgoing packets (e.g., ssh traffic) to bypass VPN--i.e., those packets should go out via default gateway of my wireless card (athn0), not via tun0.

Back in my Linux days, I could accomplish this by using iptables(8) to put a mark on packets going out through port 22, then use ip(8) to create a custom route and tell the kernel to use that route for the marked packets. Like this (I chose "1" as the mark and "table 100" as the custom route, but there is nothing special about those numbers):

Code:
iptables -t mangle -A PREROUTING -p tcp -m multiport --dport 22 -j MARK --set-mark 1
ip route add table 100 $nonvpn_default_gateway
ip rule add fwmark 1 table 100
How do I accomplish this (mark packets based on criteria such as destination port, then use custom route for marked packets) on OpenBSD? Presumably I'd mark the packets with something in pf.conf(5) then use route(8) to tell kernel which gateway to use for marked packets. Trouble is I cannot find anything in pf.conf(5) or route(8) about marked packets.

P.S. I perused all of pf.conf(5) and "tag" seems promising, but I cannot find "tag" anywhere in route(8).

Last edited by brudan; 16th January 2019 at 02:21 PM. Reason: update about "tag"
Reply With Quote
  #2   (View Single Post)  
Old 16th January 2019
jggimi's Avatar
jggimi jggimi is online now
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 6,598
Default

In PF, you can use tag/tagged and route-to. See the PF User's Guide chapter on Packet Tagging (Policy Filtering) and the pf.conf(5) man page.
Reply With Quote
  #3   (View Single Post)  
Old 16th January 2019
brudan brudan is offline
Fdisk Soldier
 
Join Date: Dec 2018
Posts: 74
Default

Thank you, jggimi. route-to looks very promising. I'll explore that and if I find a solution to my problem I'll post it here.
Reply With Quote
  #4   (View Single Post)  
Old 16th January 2019
brudan brudan is offline
Fdisk Soldier
 
Join Date: Dec 2018
Posts: 74
Default

With route-to it seems we should filter and re-route with a single rule (no need for tagging).

However, this is my first time using pf and I'm struggling a bit. I appended this to end of /etc/pf.conf:

Code:
pass out on any proto {tcp udp} from any port 22 route-to athn0
then loaded the rules with pfctl -f /etc/pf.conf. However, outgoing ssh connections are still going through VPN (running "who" on the remote machines shows that I'm logged-in with the VPN IP address).

How should I fix the filter rule so that outgoing traffic on port 22 goes through athn0, bypassing VPN? Help!

Last edited by brudan; 16th January 2019 at 04:20 PM.
Reply With Quote
  #5   (View Single Post)  
Old 16th January 2019
IdOp's Avatar
IdOp IdOp is offline
Too dumb for a smartphone
 
Join Date: May 2008
Location: twisting on the daemon's fork(2)
Posts: 846
Default

My sense is that, since port 22 is the destination port for ssh (i.e., ssh packets go to that port on the remote machine), that you need to work the keyword "to" into the pf syntax. But I don't know enough about pf to offer more than that thought.
Reply With Quote
  #6   (View Single Post)  
Old 16th January 2019
brudan brudan is offline
Fdisk Soldier
 
Join Date: Dec 2018
Posts: 74
Default

Thank you, IdOp. Based on your suggestion I tried this:
Code:
pass out on any proto {tcp udp} from any to any port 22 route-to athn0
and this:
Code:
pass out on any proto {tcp udp} to any port 22 route-to athn0
With either of these, ssh just hangs at authentication step until the connection attempt times out. (This is my first foray into pf, so please forgive me if the above is not what you meant. It's so painful being a newbie.)
Reply With Quote
  #7   (View Single Post)  
Old 16th January 2019
jggimi's Avatar
jggimi jggimi is online now
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 6,598
Default

Perhaps
Code:
pass out proto tcp to any port 22 route-to athn0
is what you need. For TCP (and UDP), there are two ports for every communication, a source port and a destination port. These port numbers are often different, and for ssh, the client selects an unused high-number port as the source port.

Edited to add: I see you've already attempted this. Try ssh with -v to see if you are actually connecting.
Reply With Quote
  #8   (View Single Post)  
Old 16th January 2019
brudan brudan is offline
Fdisk Soldier
 
Join Date: Dec 2018
Posts: 74
Default

jggimi, no luck with that either. ssh authentication attempt (with -v) hangs at "debug1: Connecting to..." then times out.

Running this as root gives me desired behavior, but only for the specified remote_ip:

Code:
athn0_gateway=$(route show | grep default | awk '{print $2}')
route add $remote_ip $athn0_gateway
It would be nice to find an elegant solution using pf and destination port, instead of my kludge above using route and destination ip
Reply With Quote
  #9   (View Single Post)  
Old 16th January 2019
jggimi's Avatar
jggimi jggimi is online now
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 6,598
Default

Connection is not occurring. I assume the TCP handshake is not completing.

Is your destination SSH server "on" your VPN network? If so, perhaps the routes are not symmetric, and the SYN and first ACK are occuring on different interfaces.

Edited to add: you can watch with tcpdump(8), to see if this is what is happening.

Last edited by jggimi; 16th January 2019 at 06:07 PM.
Reply With Quote
Old 16th January 2019
brudan brudan is offline
Fdisk Soldier
 
Join Date: Dec 2018
Posts: 74
Default

No, ssh server is not on the VPN network. My laptop is at work WiFi, openvpn client running on laptop connects to my VPN subscription (Private Internet Access), ssh server is a machine at home connected to internet via ISP.

Interestingly, even though the route command in my previous post works, if I try that same approach (based on destination ip) with pf, like this...

Code:
pass out to $remote_ip route-to athn0
or

Code:
pass out to $remote_ip route-to $athn0_gateway
...again ssh authentication hangs then times out. The only conclusion I can draw is that pf's route-to does not work as intuitively as route(8).
Reply With Quote
Old 16th January 2019
jggimi's Avatar
jggimi jggimi is online now
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 6,598
Default

I've never used route-to, and I haven't used OpenVPN in at least 25 years. So I'm not the best support resource to work up a solution using these tools. I'd be tempted to set up a second routing table, just for ssh use, if tcpdump(8) output didn't lead me to a root cause and solution.

You might ask for advice on the misc@ mailing list, it is a much larger and broader community of users than here.
Reply With Quote
Old 16th January 2019
brudan brudan is offline
Fdisk Soldier
 
Join Date: Dec 2018
Posts: 74
Default

Sounds good, jggimi. So I'll try these in this order:
1. chase my tail with route-to a bit more (will post here if I have a eureka moment)
2. tcpdump(8)
3. routing table for ssh
4. misc@ mailing list

Thank you so much for all your help! At least now I've gotten my feet wet with pf.

P.S. I'm loving OpenBSD. It's a bit painful to learn the new tools and syntax, but worthwhile for such a streamlined and elegant system. (Nothing against GNU/Linux--it has served me well for a long time, but it feels chaotic compared to OpenBSD.)
Reply With Quote
Old 16th January 2019
jggimi's Avatar
jggimi jggimi is online now
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 6,598
Default

If it were me, I'd modify the list: I'd combine PF with tcpdump and put them both first, as watching packets flow (or not) may help with PF rule changes and reduce the tail chasing. I'd put misc@ second, and go with a second routing table as a last resort. It's operationally complex for use with an end-user client application like ssh(1), because using any table other than the primary requires # route [-T rtable] exec [command ...]
Reply With Quote
Old 16th January 2019
brudan brudan is offline
Fdisk Soldier
 
Join Date: Dec 2018
Posts: 74
Default

Thank you. Yes, that seems like a better plan.

BTW I found a thread where OP is trying to do the same thing as me (except that he's interested in ports 80 and 443):
https://forums.freebsd.org/threads/b...n-ports.41600/

See posts #8 and #9 in the link. In #9, user kpa says that our approach won't work unless the route-to happens on the inbound side of filtering. I'm not sure how to accomplish that on OpenBSD (on Linux there is PREROUTING).
Reply With Quote
Old 16th January 2019
jggimi's Avatar
jggimi jggimi is online now
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 6,598
Default

If I'm reading that thread correctly, the "inbound/outbound" discussion is about an attempt to use route-to on the outbound tun() interface, and re-routing those packets to a different interface. There is some discussion in the thread of route-to on inbound traffic, but that use-case only makes sense to me if the traffic is to be forwarded, and that is not your use-case, if I understand it correctly.

Keep in mind, the PF used in FreeBSD is not the PF used in OpenBSD. IIRC FreeBSD forked PF in 2009, and the two PFs have been on separate development paths (and syntax) ever since.

---

Edited to add: the only route-to example I can find in the PF User's Guide is an example that forwards inbound traffic to a pool of outbound routers.

https://www.openbsd.org/faq/pf/pools.html#outgoing

I've looked through my copy of Peter Hansteen's The Book of PF. Unfortunately, route-to is not discussed within Peter's book.

Last edited by jggimi; 16th January 2019 at 08:40 PM. Reason: FAQ and Book
Reply With Quote
Old 17th January 2019
brudan brudan is offline
Fdisk Soldier
 
Join Date: Dec 2018
Posts: 74
Default

I asked for help on the misc mailing list and got the missing pieces for a solution.

To tag and bypass vpn in two steps as in my original question:

Code:
pass out proto tcp from (self) to port 22 tag VPNBYPASS
pass out quick tagged VPNBYPASS route-to (athn0 $athn0_gateway) nat-to (athn0)
To do it with one rule:
Code:
pass out quick proto tcp from (self) to port 22 route-to (athn0 $athn0_gateway) nat-to (athn0)
Reply With Quote
Old 17th January 2019
jggimi's Avatar
jggimi jggimi is online now
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 6,598
Default

Congratulations!
Reply With Quote
Old 18th January 2019
brudan brudan is offline
Fdisk Soldier
 
Join Date: Dec 2018
Posts: 74
Default

Thank you! And thanks for all your help. Couldn't have done it without you.
Reply With Quote
Reply

Tags
mark, pf, route, vpn, vpnbypass

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Mark Tor circuits as dirty e1-531g Guides 2 29th June 2016 07:05 AM
when fsck takes plus but can't mark filesystem clean daemonfowl OpenBSD Installation and Upgrading 3 20th September 2012 03:52 PM
'Mark-of-the-Beast' bug topples Java apps J65nko News 0 8th February 2011 01:38 AM
Mark forum read Darwimy Feedback and Suggestions 1 10th June 2008 09:23 PM
IPF: Packets Out Of Window bram85 FreeBSD Security 9 2nd June 2008 04:09 PM


All times are GMT. The time now is 03:34 PM.


Powered by vBulletin® Version 3.8.4
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
Content copyright © 2007-2010, the authors
Daemon image copyright ©1988, Marshall Kirk McKusick