Weird routing problem

[bceverly]

Hi all,

I have a bit of a problem I could use some more experienced help troubleshooting. I have an OpenBSD 5.9-release amd64 computer with two Intel NICs set up as a router/firewall in my house.

The network for the cablemodem is 10.0.0.x and my router/PC is set up on a 192.168.1.x subnet.

The problem is that I can't ping 10.0.0.1 (the default gateway for the cablemodem network) from the router or from anything in my private network.

The weird thing is that "route show" has 10.0.0.1 as the default gateway and everything else works fine (I can hit any IP address or hostname on the public Internet).

It doesn't seem to be a firewall configuration error because ping gives me "no route to host"

$ ping 10.0.0.1
PING 10.0.0.1 (10.0.0.1): 56 data bytes
ping: sendto: No route to host
ping: wrote 10.0.0.1 64 chars, ret=-1
--- 10.0.0.1 ping statistics ---
1 packets transmitted, 0 packets received, 100.0% packet loss

Any ideas? My output from "route show" is below. Happy to provide any additional diagnostic info:

Routing tables

Internet:
Destination        Gateway            Flags   Refs      Use   Mtu  Prio Iface
default            10.0.0.1           UGS      178  1615447     -     8 em0  
10.0.0/24          10.0.0.72          UC         2   111055     -     4 em0  
10.0.0.1           be:d1:65:97:3b:a3  UHLc       1    15140     -     4 em0  
10.0.0.16          ac:87:a3:18:5d:26  UHLc       0    47889     -     4 em0  
10.0.0.72          d0:50:99:64:a4:42  UHLl       0    47234     -     1 em0  
10.0.0.255         10.0.0.72          UHb        0        0     -     1 em0  
loopback           localhost          UGRS       0        3 32768     8 lo0  
localhost          localhost          UHl        3    76520 32768     1 lo0  
192.168.1/24       router             UC         6     1154     -     4 em1  
router             d0:50:99:64:a4:43  UHLl       0    24792     -     1 em1  
att-micro-cell     34:bd:fa:76:54:12  UHLc       0   452628     -     4 em1  
airport-extreme    34:12:98:01:50:f9  UHLc       0     1158     -     4 em1  
ppcbsd             00:11:24:74:9a:e8  UHLc       0     1163     -     4 em1  
192.168.1.100      80:be:05:2e:6f:86  UHLc       0    14042     -     4 em1  
192.168.1.107      74:e5:0b:50:a7:2a  UHLc       3   198052     -     4 em1  
192.168.1.161      6c:72:e7:bf:f1:02  UHLc       0     1853     -     4 em1  
192.168.1.255      router             UHb        0      242     -     1 em1  
base-address.mcast localhost          URS        0        0 32768     8 lo0

Internet6:

Destination        Gateway            Flags   Refs      Use   Mtu  Prio Iface
::/104             localhost          UGRS       0        2 32768     8 lo0  
::/96              localhost          UGRS       0        0 32768     8 lo0  
localhost          localhost          UHl       13       13 32768     1 lo0  
::127.0.0.0/104    localhost          UGRS       0        1 32768     8 lo0  
::224.0.0.0/100    localhost          UGRS       0        1 32768     8 lo0  
::255.0.0.0/104    localhost          UGRS       0        1 32768     8 lo0  
::ffff:0.0.0.0/96  localhost          UGRS       0        1 32768     8 lo0  
2002::/24          localhost          UGRS       0        1 32768     8 lo0  
2002:7f00::/24     localhost          UGRS       0        1 32768     8 lo0  
2002:e000::/20     localhost          UGRS       0        1 32768     8 lo0  
2002:ff00::/24     localhost          UGRS       0        1 32768     8 lo0  
fe80::/10          localhost          UGRS       0        1 32768     8 lo0  
fe80::1%lo0        fe80::1%lo0        UHl        0        0 32768     1 lo0  
fec0::/10          localhost          UGRS       0        1 32768     8 lo0  
ff01::/16          localhost          UGRS       0        1 32768     8 lo0  
ff01::%lo0/32      localhost          UC         0        1 32768     4 lo0  
ff02::/16          localhost          UGRS       0        1 32768     8 lo0  
ff02::%lo0/32      localhost          UC         0        1 32768     4 lo0

[rocket357]

Are you certain it isn't a firewall issue?

# pfctl -sr | grep 73.225.250.1
#
# ping 73.225.250.1
PING 73.225.250.1 (73.225.250.1): 56 data bytes
64 bytes from 73.225.250.1: icmp_seq=0 ttl=255 time=16.932 ms
64 bytes from 73.225.250.1: icmp_seq=1 ttl=255 time=7.139 ms
--- 73.225.250.1 ping statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 7.139/12.035/16.932/4.897 ms
#
#
#
# vi /etc/pf.conf # added block quick proto icmp from any to 73.225.250.1
# pfctl -f /etc/pf.conf
# pfctl -sr | grep 73.225.250.1
block drop quick inet proto icmp from any to 73.225.250.1
# ping 73.225.250.1
PING 73.225.250.1 (73.225.250.1): 56 data bytes
ping: sendto: No route to host
ping: wrote 73.225.250.1 64 chars, ret=-1
ping: sendto: No route to host
ping: wrote 73.225.250.1 64 chars, ret=-1
--- 73.225.250.1 ping statistics ---
2 packets transmitted, 0 packets received, 100.0% packet loss

What happens if you dump an explicit pass quick proto icmp from any to 10.0.0.1/32 in pf.conf?

[bceverly]

First, thank you for replying to my message.

Interesting. I had no idea that pf and ping interacted in that way. Unfortunately, I tried that and no joy:

# cat /etc/pf.conf | grep -i 10.0.0.1
pass quick proto icmp from any to 10.0.0.1/32
# ping 10.0.0.1
PING 10.0.0.1 (10.0.0.1): 56 data bytes
ping: sendto: No route to host
ping: wrote 10.0.0.1 64 chars, ret=-1
ping: sendto: No route to host
ping: wrote 10.0.0.1 64 chars, ret=-1
ping: sendto: No route to host
ping: wrote 10.0.0.1 64 chars, ret=-1
--- 10.0.0.1 ping statistics ---
3 packets transmitted, 0 packets received, 100.0% packet loss

[bceverly]

Hold the phone. I pulled my head out of... well, let's just say I thought about it.

I had put that at the BOTTOM of my /etc/pf.conf. Moved it to the top and I can ping again. Clearly I have something amiss in my firewall rules.

Thanks for getting me unstuck. I'll figure it out now I'm sure.

[bceverly]

For the curious, here's how I locked my keys in the car so to speak.

I had a macro that defined all the "unroutable" (i.e. private) subnets:

broken="224.0.0.22 127.0.0.0/8 192.168.0.0/16 172.16.0.0/12 \
        10.0.0.0/8 169.254.0.0/16 192.0.2.0/24 \
        198.51.100.0/24, 203.0.113.0/24, \
        169.254.0.0/16 0.0.0.0/8 240.0.0.0/4 255.255.255.255/32"

I then had this delightful line in my pf.conf:

block return out quick log on egress from any to { no-route $broken }

So I was explicitly telling pf to not route traffic to the 10.x.x.x subnet and tell the system there was no route to the host (hence my error message).

This used to work just fine because my old cable modem didn't have a switch and just simply assigned the public IP address that DHCP at my ISP assigned to me to my external WAN interface. Well, I had gotten a new cable modem from my ISP a few months back and it has a 4 port switch integrated to it and used the 10.x.x.x subnet.

Sorry to have bothered you folks with my own stupidity here but just in case anyone else "notices" a "problem" like this, I thought my public shaming could be put to good use.

[rocket357]

Originally Posted by bceverly

here's how I locked my keys in the car so to speak.

Happens to the best of us. I can't tell you how many times I've spent "troubleshooting" a configuration change I'd made on the fly in response to my kids doing something wonky on the network, only to discover that the route or setting I'd configured was precisely what was causing the current issue =\