|
|||
PF problem with reply-to
Hi,
I am currently running OpenBSD 4.9 as a router/firewall for my work and so far I have nearly a fully working config but there is something I cannot manage to do Here is my configuration: The server has 1 physical interface, I added a gif interface to connect it to a remote machine which is used to route most of the traffic, on this gif interface I have incoming requests I want to pass through squid. Here is my pf.conf fie: Code:
phys_if = "re0" c1_tunnel = "gif1001" c1_tunnel_dst = "xx.xx.xx.xx" c1_tunnel_src = "xx.xx.xx.xx" c1_escape = "xx.xx.xx.xx" set skip on lo0 set block-policy drop # block any packet with no match block log all # allow our own services to work pass in on $phys_if proto tcp from any to $phys_if port { ssh } synproxy state pass in on $phys_if inet proto icmp from any to $phys_if pass out on $phys_if label "system" # allow ipip traffic (gif interface) pass in on $phys_if from $c1_tunnel_dst to $c1_tunnel_src label "c1_tunnel" pass out on $phys_if from $c1_tunnel_src to $c1_tunnel_dst label "c1_tunnel" # tag incoming packets from the tunnel and from # the outside to the public ip address match in log(matches) on $c1_tunnel tag "c1" match in log(matches) on $phys_if from any to $c1_escape tag "c1" # Allow incoming packet to port 80 and redirect them to squid pass in log(matches, all) on $c1_tunnel proto tcp to port 80 \ rdr-to 127.0.0.1 port 1001 \ reply-to ($c1_tunnel 10.0.0.5) \ tagged "c1" label "c1_proxied_traffic" The result is that I cannot establish a tcp connection to port 80 for a machine behing the gif tunnel, here is what tcpdump says on this machine ( tcpdump -s 0 -vlni <interface> port 80 ): Code:
IP (tos 0x0, ttl 62, id 8175, offset 0, flags [DF], length: 60) <client_address>.21746 > <web_address>.80: S [tcp sum ok] 3993732249:3993732249(0) win 65535 <mss 1380,nop,wscale 3,sackOK,timestamp 442132554 0> IP (tos 0x0, ttl 64, id 59219, offset 0, flags [DF], length: 64) <web_address>.80 > <client_address>.21746: S [tcp sum ok] 3332347257:3332347257(0) ack 3993732250 win 16384 <mss 1440,nop,nop,sackOK,nop,wscale 3,nop,nop,timestamp 2569862135 442132554> IP (tos 0x0, ttl 62, id 8176, offset 0, flags [DF], length: 52) <client_address>.21746 > <web_address>.80: . [tcp sum ok] ack 1 win 8208 <nop,nop,timestamp 442132595 2569862135> IP (tos 0x0, ttl 62, id 8189, offset 0, flags [DF], length: 54) <client_address>.21746 > <web_address>.80: P [tcp sum ok] 1:3(2) ack 1 win 8208 <nop,nop,timestamp 442134444 2569862135> IP (tos 0x0, ttl 64, id 65354, offset 0, flags [DF], length: 58) <web_address>.80 > <client_address>.21746: P [bad tcp cksum a676 (->a9cf)!] 1:7(6) ack 3 win 2052 <nop,nop,timestamp 2569862139 442134444> IP (tos 0x0, ttl 62, id 8190, offset 0, flags [DF], length: 54) <client_address>.21746 > <web_address>.80: P [tcp sum ok] 1:3(2) ack 1 win 8208 <nop,nop,timestamp 442134770 2569862135> IP (tos 0x0, ttl 64, id 23274, offset 0, flags [DF], length: 52) <web_address>.80 > <client_address>.21746: . [bad tcp cksum b6c8 (->ba21)!] ack 3 win 2052 <nop,nop,timestamp 2569862139 442134770> IP (tos 0x0, ttl 62, id 8194, offset 0, flags [DF], length: 54) <client_address>.21746 > <web_address>.80: P [tcp sum ok] 1:3(2) ack 1 win 8208 <nop,nop,timestamp 442135222 2569862135> IP (tos 0x0, ttl 64, id 4613, offset 0, flags [DF], length: 52) <web_address>.80 > <client_address>.21746: . [bad tcp cksum b503 (->b85c)!] ack 3 win 2052 <nop,nop,timestamp 2569862140 442135222> IP (tos 0x0, ttl 62, id 8199, offset 0, flags [DF], length: 54) <client_address>.21746 > <web_address>.80: P [tcp sum ok] 1:3(2) ack 1 win 8208 <nop,nop,timestamp 442135926 2569862135> IP (tos 0x0, ttl 64, id 41419, offset 0, flags [DF], length: 52) <web_address>.80 > <client_address>.21746: . [bad tcp cksum b241 (->b59a)!] ack 3 win 2052 <nop,nop,timestamp 2569862142 442135926> IP (tos 0x0, ttl 62, id 8205, offset 0, flags [DF], length: 52) <client_address>.21746 > <web_address>.80: F [tcp sum ok] 3:3(0) ack 1 win 8208 <nop,nop,timestamp 442136662 2569862135> IP (tos 0x0, ttl 62, id 8209, offset 0, flags [DF], length: 54) <client_address>.21746 > <web_address>.80: FP [tcp sum ok] 1:3(2) ack 1 win 8208 <nop,nop,timestamp 442137134 2569862135> IP (tos 0x0, ttl 64, id 43519, offset 0, flags [DF], length: 58) <web_address>.80 > <client_address>.21746: P [bad tcp cksum 9bee (->9f47)!] 1:7(6) ack 3 win 2052 <nop,nop,timestamp 2569862145 442137134> There is only one packet from the web server (which is in fact my server since the request was redirected to squid ) which finds its way to the client and after that all the checksums are wrong, I suppose the packets are dropped by the kernel due to the bad checksum since they never reach my client application (which is curl). I tried to figure out why the checksum could be wrong but I am now out of ideas... I hope someone can help me on this. Last edited by schmurfy; 15th August 2011 at 11:12 AM. |
|
||||
It has been a while, but I think you need to replace your pf.conf fragment with something along the line of,
Code:
pass in on $phys_if from $c1_tunnel_dst to $c1_tunnel_src label "c1_tunnel" match in log(matches) on $c1_tunnel tag "c1" match in log(matches) on $phys_if from any to $c1_escape tag "c1" pass in log(matches, all) on $c1_tunnel proto tcp to port 80 \ rdr-to 127.0.0.1 port 1001 \ reply-to ($c1_tunnel 10.0.0.5) \ tagged "c1" label "c1_proxied_traffic" Code:
pass in log(matches, all) quick on $phy_if inet proto 97 \ from c1_tunnel_dst to (c1_tunnel_src) \ keep state pass out log(matches, all) quick on $phy_if inet proto 97 \ from (c1_tunnel_src) to c1_tunnel_dst \ keep state pass out log quick on $c1_tunnel inet proto tcp \ from <FARSIDE_LAN> to <NEARSIDE_LAN> port 80 \ rdr-to 127.0.0.1 port 1001 \ reply-to ($c1_tunnel 10.0.0.5) \ tag UNENCAP_TRAFFIC_FOR_LOCAL label "c1_proxied_traffic"
__________________
Never argue with an idiot. They will bring you down to their level and beat you with experience. |
|
|||
Thanks for your reply and pointing out that I forgot some ips in clear, they were not critical ones but still I prefer to not have them in clear.
I will give your suggestion a try tomorrow, I really hope it can get me pass the wall I am currently staring at. |
|
|||
After some testing it still does not work but I gathered more evidences.
I first changed these two lines as you suggested but it changed nothing: Code:
# allow ipip traffic (gif interface) pass in on $phys_if from $c1_tunnel_dst to $c1_tunnel_src label "c1_tunnel" pass out on $phys_if from $c1_tunnel_src to $c1_tunnel_dst label "c1_tunnel" That's weird for me but it looks like packets from a gif tunnels are automatically accepted by default on the physical interface, if it is not that then I am even more lost While trying stupid things I tried adding a route to the client network (where the machine issuing the http request is) telling the server which interface to use and... it worked !! I want to do all the routing with pf (in part because clients can use any private network they want so they overlap with another client) so this solution is not usable but it still worked which is a big "what the fuck ??" for me So my problem is brought down to two simple facts for now: - when I tell pf where to route the reply to it takes the right path but the checksums are wrong for all exiting packets - when I add a route in the default routing table telling the system where to route the packets to the same client it works... What is really disturbing is that the packet still reach the other end of the tunnel in both cases but the checksums are either good or bad dependeing on whether a route is present or not. It brings some questions: - Why the routing is used at all if the pf rules is set to do the routing ? (it does not work if I add a route pointing to the wrong exit) - Why adding a route "helps" pf somehow to get the right checksum God I hate these kind of weird problems I still hope I did something wrong and smeone can bring me some kind of enlightenment but it looks awfully like a bug with reply-to and gif interfaces. PS: I also tried to change the proxy rule to more closely match yours but no improvements. Edit: Silly me, I forgot the fact that even if I removed the rule for the ipip tunnel the state was still there, I removed the state and replaced the two rules with: Code:
pass in log(matches, all) quick on $phys_if inet proto 4 from $env_preprod_1_tunnel_dst to $env_preprod_1_tunnel_src label "env_preprod_1_tunnel" # pass out log(matches, all) quick on $phys_if inet proto 4 from $env_preprod_1_tunnel_src to $env_preprod_1_tunnel_dst label "env_preprod_1_tunnel" Last edited by schmurfy; 16th August 2011 at 12:45 PM. |
|
|||
Here is some more data on my current state, I checked the currently active states and they are as expected:
Code:
$ pfctl -s state all ipencap <tunnel_src> <- <tunnel_dest> MULTIPLE:MULTIPLE all tcp 127.0.0.1:1001 (<web_address>:80) <- <client_address>:22234 FIN_WAIT_2:FIN_WAIT_2 all tcp 127.0.0.1:1001 (<web_address>:80) <- <client_address>:45406 FIN_WAIT_2:FIN_WAIT_2 all tcp 127.0.0.1:1001 (<web_address>:80) <- <client_address>:54255 ESTABLISHED:ESTABLISHED The last one is a connection that was open when I typed the command showing that as far as pf is concerned everything is fine When i disconnect the client the right part transition to "CLOSING:FIN_WAIT_2" and some seconds later to "FIN_WAIT_2:FIN_WAIT_2" (I suppose the checksum problem prevent both parts to properly close the connection) |
|
|||
After banging my head on the walls for some time it looks like it is a bug with the "re" network driver, I tried on another server which uses the "em" driver and it works fine there...
|
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
Boot problem. Geometry problem? | gulanito | FreeBSD Installation and Upgrading | 0 | 3rd July 2009 03:03 AM |
UPDATING says rebuild all gettext !! HowSoEver; READ yes REPLY maybe just wait | jb_daefo | FreeBSD Ports and Packages | 4 | 8th June 2008 06:13 AM |
Zero-reply threads | anomie | Feedback and Suggestions | 5 | 22nd May 2008 02:54 PM |