DaemonForums  

Go Back   DaemonForums > OpenBSD > OpenBSD Security

OpenBSD Security Functionally paranoid!

Reply
 
Thread Tools Display Modes
  #1   (View Single Post)  
Old 10th August 2011
schmurfy schmurfy is offline
Port Guard
 
Join Date: Aug 2011
Posts: 12
Thanked 0 Times in 0 Posts
Default PF problem with reply-to

Hi,
I am currently running OpenBSD 4.9 as a router/firewall for my work and so far I have nearly a fully working config but there is something I cannot manage to do

Here is my configuration:
The server has 1 physical interface, I added a gif interface to connect it to a remote machine which is used to route most of the traffic, on this gif interface I have incoming requests I want to pass through squid.

Here is my pf.conf fie:

Code:
    phys_if = "re0"

    c1_tunnel = "gif1001"
    c1_tunnel_dst = "xx.xx.xx.xx"
    c1_tunnel_src = "xx.xx.xx.xx"
    c1_escape = "xx.xx.xx.xx"

    set skip on lo0
    set block-policy drop

    # block any packet with no match
    block log all

    # allow our own services to work
    pass in on $phys_if proto tcp from any to $phys_if port { ssh } synproxy state
    pass in on $phys_if inet proto icmp from any to $phys_if
    pass out on $phys_if label "system"

    # allow ipip traffic (gif interface)
    pass in on $phys_if from $c1_tunnel_dst to $c1_tunnel_src label "c1_tunnel"
    pass out on $phys_if from $c1_tunnel_src to $c1_tunnel_dst label "c1_tunnel"

    # tag incoming packets from the tunnel and from
    # the outside to the public ip address
    match in log(matches) on $c1_tunnel tag "c1"
    match in log(matches) on $phys_if from any to $c1_escape tag "c1"

    # Allow incoming packet to port 80 and redirect them to squid
    pass in log(matches, all) on $c1_tunnel proto tcp to port 80 \
      rdr-to 127.0.0.1 port 1001 \
      reply-to ($c1_tunnel 10.0.0.5) \
      tagged "c1" label "c1_proxied_traffic"

The result is that I cannot establish a tcp connection to port 80 for a machine behing the gif tunnel, here is what tcpdump says on this machine ( tcpdump -s 0 -vlni <interface> port 80 ):

Code:
IP (tos 0x0, ttl  62, id 8175, offset 0, flags [DF], length: 60) <client_address>.21746 > <web_address>.80: S [tcp sum ok] 3993732249:3993732249(0) win 65535 <mss 1380,nop,wscale 3,sackOK,timestamp 442132554 0>
IP (tos 0x0, ttl  64, id 59219, offset 0, flags [DF], length: 64) <web_address>.80 > <client_address>.21746: S [tcp sum ok] 3332347257:3332347257(0) ack 3993732250 win 16384 <mss 1440,nop,nop,sackOK,nop,wscale 3,nop,nop,timestamp 2569862135 442132554>

IP (tos 0x0, ttl  62, id 8176, offset 0, flags [DF], length: 52) <client_address>.21746 > <web_address>.80: . [tcp sum ok] ack 1 win 8208 <nop,nop,timestamp 442132595 2569862135>
IP (tos 0x0, ttl  62, id 8189, offset 0, flags [DF], length: 54) <client_address>.21746 > <web_address>.80: P [tcp sum ok] 1:3(2) ack 1 win 8208 <nop,nop,timestamp 442134444 2569862135>
IP (tos 0x0, ttl  64, id 65354, offset 0, flags [DF], length: 58) <web_address>.80 > <client_address>.21746: P [bad tcp cksum a676 (->a9cf)!] 1:7(6) ack 3 win 2052 <nop,nop,timestamp 2569862139 442134444>

IP (tos 0x0, ttl  62, id 8190, offset 0, flags [DF], length: 54) <client_address>.21746 > <web_address>.80: P [tcp sum ok] 1:3(2) ack 1 win 8208 <nop,nop,timestamp 442134770 2569862135>
IP (tos 0x0, ttl  64, id 23274, offset 0, flags [DF], length: 52) <web_address>.80 > <client_address>.21746: . [bad tcp cksum b6c8 (->ba21)!] ack 3 win 2052 <nop,nop,timestamp 2569862139 442134770>

IP (tos 0x0, ttl  62, id 8194, offset 0, flags [DF], length: 54) <client_address>.21746 > <web_address>.80: P [tcp sum ok] 1:3(2) ack 1 win 8208 <nop,nop,timestamp 442135222 2569862135>
IP (tos 0x0, ttl  64, id 4613, offset 0, flags [DF], length: 52) <web_address>.80 > <client_address>.21746: . [bad tcp cksum b503 (->b85c)!] ack 3 win 2052 <nop,nop,timestamp 2569862140 442135222>
IP (tos 0x0, ttl  62, id 8199, offset 0, flags [DF], length: 54) <client_address>.21746 > <web_address>.80: P [tcp sum ok] 1:3(2) ack 1 win 8208 <nop,nop,timestamp 442135926 2569862135>
IP (tos 0x0, ttl  64, id 41419, offset 0, flags [DF], length: 52) <web_address>.80 > <client_address>.21746: . [bad tcp cksum b241 (->b59a)!] ack 3 win 2052 <nop,nop,timestamp 2569862142 442135926>

IP (tos 0x0, ttl  62, id 8205, offset 0, flags [DF], length: 52) <client_address>.21746 > <web_address>.80: F [tcp sum ok] 3:3(0) ack 1 win 8208 <nop,nop,timestamp 442136662 2569862135>
IP (tos 0x0, ttl  62, id 8209, offset 0, flags [DF], length: 54) <client_address>.21746 > <web_address>.80: FP [tcp sum ok] 1:3(2) ack 1 win 8208 <nop,nop,timestamp 442137134 2569862135>
IP (tos 0x0, ttl  64, id 43519, offset 0, flags [DF], length: 58) <web_address>.80 > <client_address>.21746: P [bad tcp cksum 9bee (->9f47)!] 1:7(6) ack 3 win 2052 <nop,nop,timestamp 2569862145 442137134>

There is only one packet from the web server (which is in fact my server since the request was redirected to squid ) which finds its way to the client and after that all the checksums are wrong, I suppose the packets are dropped by the kernel due to the bad checksum since they never reach my client application (which is curl).
I tried to figure out why the checksum could be wrong but I am now out of ideas...

I hope someone can help me on this.

Last edited by schmurfy; 15th August 2011 at 11:12 AM.
Reply With Quote
  #2   (View Single Post)  
Old 13th August 2011
s2scott's Avatar
s2scott s2scott is offline
Package Pilot
 
Join Date: May 2008
Location: Toronto, Ontario Canada
Posts: 198
Thanked 9 Times in 8 Posts
Default

Code:
c1_tunnel_src = "87.xx.xx.50"
c1_escape = "87.xx.xx.179"
If these are your REAL IP addresses, for your own protection, you should more properly MASK them before publishing them here.
__________________
Never argue with an idiot. They will bring you down to their level and beat you with experience.
Reply With Quote
  #3   (View Single Post)  
Old 13th August 2011
s2scott's Avatar
s2scott s2scott is offline
Package Pilot
 
Join Date: May 2008
Location: Toronto, Ontario Canada
Posts: 198
Thanked 9 Times in 8 Posts
Default

It has been a while, but I think you need to replace your pf.conf fragment with something along the line of,

Code:
pass in on $phys_if from $c1_tunnel_dst to $c1_tunnel_src label "c1_tunnel"

match in log(matches) on $c1_tunnel tag "c1"
match in log(matches) on $phys_if from any to $c1_escape tag "c1"

pass in log(matches, all) on $c1_tunnel proto tcp to port 80 \
      rdr-to 127.0.0.1 port 1001 \
      reply-to ($c1_tunnel 10.0.0.5) \
      tagged "c1" label "c1_proxied_traffic"
with

Code:
pass in log(matches, all) quick on $phy_if inet proto 97 \
   from c1_tunnel_dst to (c1_tunnel_src) \
     keep state

pass out log(matches, all) quick on $phy_if inet proto 97 \
   from (c1_tunnel_src) to c1_tunnel_dst \
     keep state

pass out log quick on $c1_tunnel inet proto tcp \
 from <FARSIDE_LAN> to <NEARSIDE_LAN> port 80 \
   rdr-to 127.0.0.1 port 1001 \
      reply-to ($c1_tunnel 10.0.0.5) \
         tag UNENCAP_TRAFFIC_FOR_LOCAL label "c1_proxied_traffic"
You must first pass in/out the gif encap packets (which i think is proto 97*), then pass in/out the deencapsulated tcp packets. (* Don't confuse proto IDs with ports.)
__________________
Never argue with an idiot. They will bring you down to their level and beat you with experience.
Reply With Quote
  #4   (View Single Post)  
Old 15th August 2011
schmurfy schmurfy is offline
Port Guard
 
Join Date: Aug 2011
Posts: 12
Thanked 0 Times in 0 Posts
Default

Thanks for your reply and pointing out that I forgot some ips in clear, they were not critical ones but still I prefer to not have them in clear.

I will give your suggestion a try tomorrow, I really hope it can get me pass the wall I am currently staring at.
Reply With Quote
  #5   (View Single Post)  
Old 16th August 2011
schmurfy schmurfy is offline
Port Guard
 
Join Date: Aug 2011
Posts: 12
Thanked 0 Times in 0 Posts
Default

After some testing it still does not work but I gathered more evidences.

I first changed these two lines as you suggested but it changed nothing:
Code:
    # allow ipip traffic (gif interface)
    pass in on $phys_if from $c1_tunnel_dst to $c1_tunnel_src label "c1_tunnel"
    pass out on $phys_if from $c1_tunnel_src to $c1_tunnel_dst label "c1_tunnel"
I then checked "pfctl -vs rules" and noticed that these rules matched no packets so I tried removing them and everything still works the same without them Oo
That's weird for me but it looks like packets from a gif tunnels are automatically accepted by default on the physical interface, if it is not that then I am even more lost

While trying stupid things I tried adding a route to the client network (where the machine issuing the http request is) telling the server which interface to use and... it worked !!

I want to do all the routing with pf (in part because clients can use any private network they want so they overlap with another client) so this solution is not usable but it still worked which is a big "what the fuck ??" for me

So my problem is brought down to two simple facts for now:
- when I tell pf where to route the reply to it takes the right path but the checksums are wrong for all exiting packets
- when I add a route in the default routing table telling the system where to route the packets to the same client it works...

What is really disturbing is that the packet still reach the other end of the tunnel in both cases but the checksums are either good or bad dependeing on whether a route is present or not.


It brings some questions:
- Why the routing is used at all if the pf rules is set to do the routing ? (it does not work if I add a route pointing to the wrong exit)
- Why adding a route "helps" pf somehow to get the right checksum

God I hate these kind of weird problems
I still hope I did something wrong and smeone can bring me some kind of enlightenment but it looks awfully like a bug with reply-to and gif interfaces.

PS: I also tried to change the proxy rule to more closely match yours but no improvements.

Edit: Silly me, I forgot the fact that even if I removed the rule for the ipip tunnel the state was still there, I removed the state and replaced the two rules with:
Code:
pass in log(matches, all) quick on $phys_if inet proto 4 from $env_preprod_1_tunnel_dst to $env_preprod_1_tunnel_src label "env_preprod_1_tunnel"
# pass out log(matches, all) quick on $phys_if inet proto 4 from $env_preprod_1_tunnel_src to $env_preprod_1_tunnel_dst label "env_preprod_1_tunnel"
but the behavior is still the same as described in this post

Last edited by schmurfy; 16th August 2011 at 12:45 PM.
Reply With Quote
  #6   (View Single Post)  
Old 16th August 2011
schmurfy schmurfy is offline
Port Guard
 
Join Date: Aug 2011
Posts: 12
Thanked 0 Times in 0 Posts
Default

Here is some more data on my current state, I checked the currently active states and they are as expected:

Code:
$ pfctl -s state
all ipencap <tunnel_src> <- <tunnel_dest>       MULTIPLE:MULTIPLE
all tcp 127.0.0.1:1001 (<web_address>:80) <- <client_address>:22234       FIN_WAIT_2:FIN_WAIT_2
all tcp 127.0.0.1:1001 (<web_address>:80) <- <client_address>:45406       FIN_WAIT_2:FIN_WAIT_2
all tcp 127.0.0.1:1001 (<web_address>:80) <- <client_address>:54255       ESTABLISHED:ESTABLISHED
As I suspected a state was created for the tunnel without me requiring it which is fine by me, the others are my attempts to connect from the client network.
The last one is a connection that was open when I typed the command showing that as far as pf is concerned everything is fine

When i disconnect the client the right part transition to "CLOSING:FIN_WAIT_2" and some seconds later to "FIN_WAIT_2:FIN_WAIT_2" (I suppose the checksum problem prevent both parts to properly close the connection)
Reply With Quote
  #7   (View Single Post)  
Old 26th August 2011
schmurfy schmurfy is offline
Port Guard
 
Join Date: Aug 2011
Posts: 12
Thanked 0 Times in 0 Posts
Default

After banging my head on the walls for some time it looks like it is a bug with the "re" network driver, I tried on another server which uses the "em" driver and it works fine there...
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Boot problem. Geometry problem? gulanito FreeBSD Installation and Upgrading 0 3rd July 2009 03:03 AM
UPDATING says rebuild all gettext !! HowSoEver; READ yes REPLY maybe just wait jb_daefo FreeBSD Ports and Packages 4 8th June 2008 06:13 AM
Zero-reply threads anomie Feedback and Suggestions 5 22nd May 2008 02:54 PM


All times are GMT. The time now is 05:53 AM.


Powered by vBulletin® Version 3.8.4
Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content copyright © 2007-2010, the authors
Daemon image copyright ©1988, Marshall Kirk McKusick