|
OpenBSD Security Functionally paranoid! |
|
Thread Tools | Display Modes |
|
|||
Issue with WireGuard and routing domains on OBSD 7.0
TL;DR: WireGuard acting as a client under fully-patched OBSD 7.0 on an RPi4 works perfectly when using a single routing domain, but then fails when the external interface is put into a separate routing domain - rdomain 1. tshark shows handshake initiation packets leaving the external interface but no response being received from the server. Has anyone successfully implemented a WireGuard client with the latest release of OBSD and split routing domains?
__________________ I've been running WireGuard for some time now on a server connected to the Internet, initially using wireguard-go and more recently using hostname.wg0 under OBSD 7.0. The sole purpose of the server is to turn traffic around and send it back out to the Internet when I'm abroad and make the traffic appear to be coming from the UK. Standard stuff to be able to access UK streaming services. It all works perfectly with a variety of clients: WireGuard on Android, Windows, Linux and various appliances. It's also a lot faster than using OpenVPN which I also have running on the server as a backup, although I'll probably decommission that soon. I've recently been playing with a WireGuard client gateway using OBSD 7.0 on a Raspberry Pi 4. In its basic form using a single default routing domain, WireGuard works fine. I use different priority routes for direct traffic to the VPN router and all other traffic intended to pass through the wg0 tunnel. tshark shows WireGuard handshakes and encrypted traffic passing through the external interface bse0 exactly as expected. Next step... To introduce greater separation between the direct and tunneled routes to the server I have then made two modifications: I've added an rdomain 1 line to the start of the hostname.bse0 external interface file and wgrtable 1 to the wgpeer line in the hostname.wg0 file. (In addition I've modified sshd_config to listen to addresses from rdomains 0 and 1, although that's not pertinent to WireGuard behaviour. I've also not bothered with DNS mods at this stage as I'm testing using pings with IP addresses and dig with a specified external DNS.) On reboot, I've made wg0 the default interface for rdomain 0 traffic. With that, WireGuard has stopped working on the RPi4. tshark shows outbound handshake requests going out to the server but no response from the server. If I switch back to single-domain operation everything works OK again. This seems to suggest that the handshake request has become corrupted such that the server doesn't recognise it, but I would love someone else to find some other explanation! Has anyone any experience of WireGuard on OBSD 7.0 running with split routing domains? Or is anyone willing to have a go themselves at replicating the situation? It doesn't need to be an RPi: any host system would do! ___________________ Here's a copy of the tshark output monitoring the external interface for traffic to/from the WireGuard server. With a single default routing domain: Code:
-bash-5.1$ doas tshark -f 'host x.x.x.x' -i bse0 Capturing on 'bse0' 1 0.000000 192.168.1.125 ? x.x.x.x WireGuard 190 Handshake Initiation, sender=0xFBFAD38C 2 0.004860 x.x.x.x ? 192.168.1.125 WireGuard 134 Handshake Response, sender=0x47722CB5, receiver=0xFBFAD38C 3 0.006097 192.168.1.125 ? x.x.x.x WireGuard 170 Transport Data, receiver=0x47722CB5, counter=0, datalen=96 4 0.006649 x.x.x.x ? 192.168.1.125 WireGuard 74 Keepalive, receiver=0xFBFAD38C, counter=0 5 0.023525 x.x.x.x ? 192.168.1.125 WireGuard 170 Transport Data, receiver=0xFBFAD38C, counter=1, datalen=96 6 1.005879 192.168.1.125 ? x.x.x.x WireGuard 170 Transport Data, receiver=0x47722CB5, counter=1, datalen=96 7 1.024303 x.x.x.x ? 192.168.1.125 WireGuard 170 Transport Data, receiver=0xFBFAD38C, counter=2, datalen=96 8 2.005833 192.168.1.125 ? x.x.x.x WireGuard 170 Transport Data, receiver=0x47722CB5, counter=2, datalen=96 9 2.020512 x.x.x.x ? 192.168.1.125 WireGuard 170 Transport Data, receiver=0xFBFAD38C, counter=3, datalen=96 10 3.005815 192.168.1.125 ? x.x.x.x WireGuard 170 Transport Data, receiver=0x47722CB5, counter=3, datalen=96 11 3.022067 x.x.x.x ? 192.168.1.125 WireGuard 170 Transport Data, receiver=0xFBFAD38C, counter=4, datalen=96 12 4.005796 192.168.1.125 ? x.x.x.x WireGuard 170 Transport Data, receiver=0x47722CB5, counter=4, datalen=96 13 4.020740 x.x.x.x ? 192.168.1.125 WireGuard 170 Transport Data, receiver=0xFBFAD38C, counter=5, datalen=96 14 5.005779 192.168.1.125 ? x.x.x.x WireGuard 170 Transport Data, receiver=0x47722CB5, counter=5, datalen=96 15 5.020599 x.x.x.x ? 192.168.1.125 WireGuard 170 Transport Data, receiver=0xFBFAD38C, counter=6, datalen=96 16 6.005758 192.168.1.125 ? x.x.x.x WireGuard 170 Transport Data, receiver=0x47722CB5, counter=6, datalen=96 17 6.020418 x.x.x.x ? 192.168.1.125 WireGuard 170 Transport Data, receiver=0xFBFAD38C, counter=7, datalen=96 18 7.005756 192.168.1.125 ? x.x.x.x WireGuard 170 Transport Data, receiver=0x47722CB5, counter=7, datalen=96 19 7.020781 x.x.x.x ? 192.168.1.125 WireGuard 170 Transport Data, receiver=0xFBFAD38C, counter=8, datalen=96 20 8.005738 192.168.1.125 ? x.x.x.x WireGuard 170 Transport Data, receiver=0x47722CB5, counter=8, datalen=96 21 8.020502 x.x.x.x ? 192.168.1.125 WireGuard 170 Transport Data, receiver=0xFBFAD38C, counter=9, datalen=96 22 9.005714 192.168.1.125 ? x.x.x.x WireGuard 170 Transport Data, receiver=0x47722CB5, counter=9, datalen=96 23 9.027533 x.x.x.x ? 192.168.1.125 WireGuard 170 Transport Data, receiver=0xFBFAD38C, counter=10, datalen=96 24 19.025513 192.168.1.125 ? x.x.x.x WireGuard 74 Keepalive, receiver=0x47722CB5, counter=10 ^C24 packets captured Code:
-bash-5.1$ doas tshark -f 'host x.x.x.x' -i bse0 Capturing on 'bse0' 1 0.000000 192.168.1.125 ? x.x.x.x WireGuard 190 Handshake Initiation, sender=0xF8014803 2 5.219900 192.168.1.125 ? x.x.x.x WireGuard 190 Handshake Initiation, sender=0x65088075 3 10.419816 192.168.1.125 ? x.x.x.x WireGuard 190 Handshake Initiation, sender=0x9A42E713 4 15.599741 192.168.1.125 ? x.x.x.x WireGuard 190 Handshake Initiation, sender=0xDD48EACC 5 20.689660 192.168.1.125 ? x.x.x.x WireGuard 190 Handshake Initiation, sender=0xAB5DDADD 6 25.819580 192.168.1.125 ? x.x.x.x WireGuard 190 Handshake Initiation, sender=0x0226D5DE ^C6 packets captured Code:
wg0: flags=80c3<UP,BROADCAST,RUNNING,NOARP,MULTICAST> mtu 1420 description: WireGuard interface index 6 priority 0 llprio 3 wgport 36559 wgrtable 1 wgpubkey <client pubkey> wgpeer <server pubkey> wgendpoint x.x.x.x 51820 tx: 0, rx: 0 wgaip 0.0.0.0/0 groups: wg inet 10.0.0.11 netmask 0xffffff00 broadcast 10.0.0.255 Last edited by Reeshar; 7th February 2022 at 11:26 AM. Reason: Added information |
|
||||
Hello, and welcome!
I tested but never implemented rdomain separation with WireGuard, as I need to be able to have my OpenBSD client disable / enable the VPN easily by just setting the wg(4) driver down or up. With routing domains, the VPN needed to be used full-time. OpenBSD developer solene@ has a blog entry with example provisioning: https://dataswamp.org/~solene/2021-1...uard-exit.html |
|
|||
Yes, I've previously read that and also:
https://codimd.laas.fr/s/NMc3qt5PQ I don't see anything in those that conflicts with what I've done. Unfortunately there's been very little written on this subject from real practical experience with the latest OBSD. For example Solène writes: This guide should work from OpenBSD 6.9. (My use of bold). Hence I'm appealing to others to give it a whirl! I've also tried testing with/without rdomain setups with a free 7-day WireGuard setup with vpnjantit: (https://www.vpnjantit.com/free-wireguard) with similar issues. It works without rdomains but not with. |
|
|||
Great! Thanks for trying that jggimi.
But I have to say I am completely gob-smacked (to use an old English expression)! I've started up my RPi config today - exactly the same as had failed before - and it is now working. Working to my local server and working to a remote WireGuard server (vpnjantit). I haven't changed anything!! I really don't understand because I had rebooted this config several times in the past to no avail. And now on booting it's worked. For me, the default route for rdomain 1 is already present without needing to add it because I'm using dhcpleased on that interface: Code:
-bash-5.1$ route -n -T 1 show -inet Routing tables Internet: Destination Gateway Flags Refs Use Mtu Prio Iface default 192.168.1.1 UGS 1 7 - 8 bse0 192.168.1/24 192.168.1.125 UCn 2 139 - 4 bse0 192.168.1.1 00:e0:67:21:cd:21 UHLch 1 151 - 3 bse0 192.168.1.125 e4:5f:01:4a:00:b9 UHLl 0 3926 - 1 bse0 192.168.1.139 84:39:be:6d:c9:a7 UHLc 1 7 - 3 bse0 192.168.1.255 192.168.1.125 UHb 0 59 - 1 bse0 Code:
doas route add default -link -iface wg0 Code:
-bash-5.1$ route -n show -inet Routing tables Internet: Destination Gateway Flags Refs Use Mtu Prio Iface default link#6 ULS 5 3405 - 8 wg0 224/4 127.0.0.1 URS 0 106 32768 8 lo0 127/8 127.0.0.1 UGRS 0 0 32768 8 lo0 127.0.0.1 127.0.0.1 UHhl 1 2 32768 1 lo0 192.168.6/24 192.168.6.111 UCn 0 0 - 4 wg0 192.168.6.111 wg0 UHl 0 90 - 1 wg0 192.168.6.255 192.168.6.111 UHb 0 0 - 1 wg0 192.168.101/24 192.168.101.1 UCn 1 0 - 4 axen0 192.168.101.1 00:50:b6:ec:c0:25 UHLl 0 64 - 1 axen0 192.168.101.100 f8:e4:3b:4f:ad:c2 UHLc 3 3247 - 3 axen0 192.168.101.255 192.168.101.1 UHb 0 1 - 1 axen0 I'll investigate further with the info you've provided! |
|
|||
And now having moved some physical network connections around it's not working again...
Totally weird. And yet I can ping the Internet from rdomain 1... |
|
|||
And now I've finally sorted it - I think anyway. It's the clock!
Normally an RPi sets its clock on boot-up as it doesn't have a built-in RTC. But when you create a separate rdomain 1 for the Internet side of the Pi, ntpd can't access any time servers. So the clock isn't set and WireGuard doesn't work. Remove the domain separation and ntpd can once again access time servers and so WireGuard works. I've tested this by using rdate to set the time against a cloudflare time server: Code:
doas route -T 1 exec rdate 162.159.200.123 Good grief that took a while to work out!! I've just got to sort out starting ntpd using rtable 1. |
|
||||
Sorry, you were already aware of route exec. The easiest way to set this at boot time would be to provision the ntpd_rtable flag via rcctl(8). See the rc.d(8) man page for a description of this flag. There's an example of reviewing this flag in the rcctl(8) man page.
Last edited by jggimi; 11th February 2022 at 04:13 AM. Reason: added last sentence |
|
|||
Yup, that's exactly what I've done. I used:
Code:
# rcctl set ntpd rtable 1 Code:
ntpd_rtable=1 Presumably I could have added the flag manually. The only remaining thing that puzzles me slightly is dhpleased. If I do Code:
# ps ax -O rtable So should I move dhcpleased to rtable 1 as with ntpd? Or does dhcpleased work across rdomains? Once I've sorted all of this, I put together a separate post on how to run WireGuard with rdomains on a RPi in case anyone else in interested. Incidentally, although I show commands as root, I actually use doas as I always have done. But what I didn't realise was that ifconfig gives different details if you execute it as a user and if you execute it as root. So to get a full description of the wg0 interface you need to execute ifconfig as root: Code:
bash-5.1$ ifconfig wg0 wg0: flags=80c3<UP,BROADCAST,RUNNING,NOARP,MULTICAST> mtu 1420 description: vpnjantit WireGuard connection index 6 priority 0 llprio 3 wgport 16146 wgrtable 1 groups: wg egress inet 192.168.6.111 netmask 0xffffff00 broadcast 192.168.6.255 Code:
bash-5.1$ doas ifconfig wg0 wg0: flags=80c3<UP,BROADCAST,RUNNING,NOARP,MULTICAST> mtu 1420 description: vpnjantit WireGuard connection index 6 priority 0 llprio 3 wgport 16146 wgrtable 1 wgpubkey DRWTID5iUrkEDczOv+Y1KavaUWRnlDCmQMZoYjKyWWE= wgpeer DiUst9llpM3ROfXwHueAFu+seOqUw8ihqIKyiKLqbmA= wgendpoint 188.119.148.113 1024 tx: 697980, rx: 1743164 last handshake: 63 seconds ago wgaip ::/0 wgaip 0.0.0.0/0 groups: wg egress inet 192.168.6.111 netmask 0xffffff00 broadcast 192.168.6.255 Last edited by Reeshar; 11th February 2022 at 08:19 AM. |
|
||||
|
|
|||
The key is only valid for another day or so. vpnjantit allocates free WireGuard for 7 days at a time and I'm only using for testing. It's not my prime WireGuard details. Hence I'm not bothered about exposing the details.
Good to know about DHCP. I thought the process running in rdomain 0 might prevent it from accessing the external interface. |
Thread Tools | |
Display Modes | |
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
pf - wireguard | vazaro | OpenBSD General | 10 | 1st November 2021 05:19 PM |
Screenshots of WireGuard Android app | J65nko | General software and network | 1 | 5th June 2021 12:00 AM |
FreeBSD port of Wireguard Code Quality Disaster | bashrules | News | 3 | 27th March 2021 08:34 AM |
WireGuard: replacement for IPsec | e1-531g | News | 15 | 16th March 2021 09:08 AM |
ARP Issue: Bridging, Routing, and FreeBSD LAGGs | jasonvp | FreeBSD Security | 14 | 5th December 2015 05:35 PM |