pf - wireguard

[vazaro]

Hello,

My VPN provider do openvpn or wierguard. Of course I haven't any shell access on
it.

My goal is to use wireguard.
In a first step I do a testing network

My network is

Provider --- internet --- firewall --- PC

firewall and PC are virtualbox computers

The network firewall configuration is
vio0 - 192.168.150.31 - outside
vio1 - 192.168.50.242 - inside

The PC network configuration is
vio0 - 192.168.50.150

The pf.conf (of course it's a minimalist pfi for testing) is

firewall#cat /etc/pf.conf
pass out quick log on vio0 from (vio1:network) nat-to (vio0)
pass out quick log on wg0 from (vio1:network) nat-to (wg)

Without wireguard, my PC access anything on internet. So all is OK

With wireguard, on my PC, the traceroute is OK, I pass into wg. The DNS is OK.

But I found a problem

I cannot access every web pages. I can access some https pages (j2b.fr for examp
le) but not other (duckduckgo.com)
firefox says "The connection has timed out"

I suppose I forget a rule in pf.conf but I cannot find it.

Can you help me please ?

firewall# wg-quick up /etc/wireguard/wg0.conf
[#] ifconfig wg0 create description wg-quick: wg0
[#] wg setconf wg0 /dev/fd/63
[#] ifconfig wg0 inet 10.140.1.31 alias
[#] ifconfig wg0 mtu 1420
[#] ifconfig wg0 up
[#] cp /etc/resolv.conf /etc/resolv.conf.wg-quick-backup.wg0
[#] printf nameserver %s\n 192.168.150.177
[#] route -q -n add -inet 0.0.0.0/1 -iface 10.140.1.31
[#] route -q -n add -inet 128.0.0.0/1 -iface 10.140.1.31
[#] route -q -n delete -inet X.X.X.X
[#] route -q -n add -inet X.X.X.X -gateway 192.168.150.242
[+] Backgrounding route monitor

firewall# wg show
interface: wg0
  public key: <removed>
  private key: (hidden)
  listening port: 20180

peer: <removed>
  endpoint: X.X.X.X:48040
  allowed ips: 0.0.0.0/0
  latest handshake: 20 seconds ago
  transfer: 92 B received, 360 B sent
  persistent keepalive: every 30 seconds

firewall# ifconfig wg0
wg0: flags=80c3<UP,BROADCAST,RUNNING,NOARP,MULTICAST> mtu 1420
        description: wg-quick: wg0
        index 10 priority 0 llprio 3
        wgport 20180
        wgpubkey <removed>
        wgpeer <removed>
                wgpka 30 (sec)
                wgendpoint X.X.X.X 48040
                tx: 392, rx: 92
                last handshake: 36 seconds ago
                wgaip 0.0.0.0/0
        groups: wg
        inet 10.140.1.31 netmask 0xff000000 broadcast 10.255.255.255

Thanks

[vns3]

I am pretty sure nat-to should be part of a match rule, not a pass rule, and then you will need a pass rule as well to allow the traffic.

(Please disreguard, this couldn't be wronger lol!)

[jggimi]

In my networks I don't use wireguard-tools, as wg(4) is a kernel driver and I provision WireGuard with hostname.if(5) only. So I may be missing something, but your problem behaves like and appears to me like a possible routing issue. The description of your inner / outer subnets (192.168...) in your post doesn't match the routes added in the wg-quick script (10.140...).

[vazaro]

Hello jggimi,

Ok I swith to wg and hostname.wg0.

My etc/hostname.wg0 is

wgkey <removed>
wgpeer <removed> wgaip 0.0.0.0/0 wgendpoint X.X.X.X 48040

inet 10.140.1.45 255.255.255.0 NONE
!route add -priority 2 X.X.X.X 192.168.150.242
!route add 0.0.0.0/1 10.140.1.45 
!route add 128.0.0.0/1 10.140.1.45 
up

The 10.140 IP is the IP my provider gives for the tunnel
The 192.168.150 IP is my working IP

In fact my configuration is

internet --- public IP - firewall - 192.168.150.242 ----- 192.168.150.31 - wireguard firewall - 192.168.50.242 --- 192.168.50.150 - working station

And when the VPN is up it becomes

internet --- 10.140.1.45 - wireguard firewall - 192.168.50.242 --- 192.168.50.150 - working station

My route are

Destination        Gateway            Flags   Refs      Use   Mtu  Prio Iface
default            192.168.150.242    UGS        0        1     -     8 vio0
0/1                10.140.1.45        UGS        2       85     -     8 wg0
128/1              10.140.1.45        UGS        3       73     -     8 wg0
224/4              127.0.0.1          URS        0        0 32768     8 lo0
10.140.1/24        10.140.1.45        UCn        0        0     -     4 wg0
10.140.1.45        wg0                UHhl       2      263     -     1 wg0
10.140.1.255       10.140.1.45        UHb        0        0     -     1 wg0
127/8              127.0.0.1          UGRS       0        0 32768     8 lo0
127.0.0.1          127.0.0.1          UHhl       1        2 32768     1 lo0
X.X.X.X        192.168.150.242    UGHS       1        1     -     2 vio0
192.168.50/24      192.168.50.242     UCn        2        0     -     4 vio1

My problem persists

[jggimi]

I see two errors with loopback. First, loopback should not be routed. Second, the loopback subnet has the wrong netmask. It is 128/8 (netmask 255.0.0.0). You've routed 128/1, which is netmask 128.0.0.0.

The inside of the tunnel appears to be incorrectly routed. 0.0.0.0/1 is not the default route, you should use 0.0.0.0/0 or the keyword "default". The inner route should also have a higher priority than your non-VPN default route, such as priority 7. This way, the tunnel will be used when it is available, but your non-VPN default route will be used if the tunnel is not in operation, such as when you intentionally set it "down" with ifconfig(8).

The outside of the tunnel appears to be routed correctly. It has the highest routing priority.

[vazaro]

Ok, I tried to simply my route table.
Now, i have

Destination        Gateway            Flags   Refs      Use   Mtu  Prio Iface
default            10.140.1.45        UGS        0      194     -     2 wg0
10.140.1/24        10.140.1.45        UCn        0        0     -     4 wg0
10.140.1.45        wg0                UHhl       1        2     -     1 wg0
10.140.1.255       10.140.1.45        UHb        0        0     -     1 wg0
127/8              127.0.0.1          UGS        0        0 32768     8 lo0
127.0.0.1          127.0.0.1          UHhl       1       16 32768     1 lo0
XX.XX.XX.XX        192.168.150.242    UGHS       1       12     -     2 vio0

But I have always the same problem.
Why can I access https://google.fr and not https://duckduckgo.com for example from my PC ?

Whereas, when I try to access duckduckgo page from the firewall directly it's OK

[jggimi]

The priority for your default route should not be the same as your outer tunnel.

[jggimi]

Here's my laptop's routing table. Note that I have two default routes at different priorities: 192.168.99.1 is my inner WireGuard tunnel endpoint, and is set to -priority 7. The DHCP auto-configured default route is priority 12.

Routing tables

Internet:
Destination        Gateway            Flags   Refs      Use   Mtu  Prio Iface
default            192.168.99.1       UGS        5     1356     -     7 wg0  
default            192.168.1.254      UGS        5       49     -    12 iwn0 
<my.server>        192.168.1.254      UGHS       0        0     -     2 iwn0 
127/8              127.0.0.1          UGRS       0        0 32768     8 lo0  
127.0.0.1          127.0.0.1          UHhl       3      444 32768     1 lo0  
192.168.1/24       192.168.1.74       UCn        1     2436     -     8 iwn0 
192.168.1.74       8c:70:5a:c4:39:bc  UHLl       0     3761     -     1 iwn0 
192.168.1.254      dc:7f:a4:6d:3b:f1  UHLch      2     1372     -     7 iwn0 
192.168.1.255      192.168.1.74       UHb        0        0     -     1 iwn0 
192.168.99/24      192.168.99.3       UCn        1        0     -     4 wg0  
192.168.99.1       link#0             UHch       1        2     -     3 wg0  
192.168.99.3       wg0                UHl        0       29     -     1 wg0  
192.168.99.255     192.168.99.3       UHb        0        0     -     1 wg0

[jggimi]

One thing you can do to check your routing configuration is to use traceroute(1). If you are not using the correct gateway to any remote address, you'll know it from the first response.

[vazaro]

OK, I changed my route table like this

Destination        Gateway            Flags   Refs      Use   Mtu  Prio Iface
default            10.140.1.45        UGS        5        8     -     8 wg0
224/4              127.0.0.1          URS        0        0 32768     8 lo0
10.140.1/24        10.140.1.45        UCn        0        0     -     4 wg0
10.140.1.45        wg0                UHhl       1       60     -     1 wg0
10.140.1.255       10.140.1.45        UHb        0        0     -     1 wg0
127/8              127.0.0.1          UGRS       0        0 32768     8 lo0
127.0.0.1          127.0.0.1          UHhl       1        2 32768     1 lo0
185.5.46.25        192.168.150.242    UGHS       1        1     -    12 vio0

On the PC a traceroute to google.fr is

PC#traceroute google.fr
 1  192.168.50.242 (192.168.50.242)  0.999 ms  0.189 ms  0.133 ms
 2  10.140.0.1 (10.140.0.1)  52.082 ms  53.121 ms  53.89 ms
 3  aaaaa.net (AA.BB.CC.DD)  53.36 ms  53.202 ms  54.783 ms <- VPN PROVIDER
 4  bbbbb.net (EE.FF.GG.HH)  54.103 ms  53.976 ms  53.511 ms <- VPN PROVIDER
 5  outside
...
And so on still google. OK

And a traceroute to duckduckgo.com is

PC#traceroute duckduckgo.com
 1  192.168.50.242 (192.168.50.242)  0.276 ms  0.184 ms  0.148 ms
 2  10.140.0.1 (10.140.0.1)  52.75 ms  54.144 ms  52.801 ms
 3  aaaaa.net (AA.BB.CC.DD)  54.457 ms  53.062 ms  53.515 ms <- VPN PROVIDER
 4  ccccc.net (II.JJ.KK.MM)  54.079 ms  53.278 ms  53.969 ms <- VPN PROVIDER
 5  outside
...
And so on still google. OK

So I can route and google and duckduckgo but firefox can find the page google and not duckduckgo.

If I do a tcpdump, the problem seems be in the pf rules, the ssl/tls transaction changes in the case with or without wireguard.

[jggimi]

The lower the -priority number, the higher the priority of the route. Whenever there are multiple active routes to the same destination, the higher priority (or lower -priority number) will be used. According to your most recent routing table, your default route is set to -priority 4, which is HIGHER than the route to your WireGuard server -- the outside of the VPN tunnel, which is set to -priority 12.

In my configuration, the -priority for the outside of the tunnel to my WireGuard server is -priority 2. This is the HIGHEST priority that can be set with route(8), as -priority 1 is reserved for assignment by the kernel. My two default routes are set at different priorities: the "through the VPN" default route is set to -priority 7, which is higher than the "use when the VPN is down" default route, which is set to -priority 12.