DaemonForums  

Go Back   DaemonForums > OpenBSD > OpenBSD Security

OpenBSD Security Functionally paranoid!

Reply
 
Thread Tools Display Modes
  #1   (View Single Post)  
Old 13th September 2016
rbigm101 rbigm101 is offline
New User
 
Join Date: Sep 2016
Posts: 8
Default MacVTap VEPA with OpenBSD router/firewall, need bridge to reflect on same segment

Hello community! Long time thread reader, first time poster. Here is what I would like some help with if someone is available to do so.

I'm using OpenBSD 5.0 i386

I had followed this guide posted below, and I had successfully set up a couple VEPA connections using MacVTap on my Debian KVM server running libvirt. If you are unfamiliar with this type of VM connection, it does not allow the VM's or the host to talk to each other, all frames get sent to the LAN directly.

The advantage of this is that I can use my existing firewall rules in pf, and simply add the VM to the appropriate table and enjoy the benefits of OpenBSD security. All while still having my VM's isolated from the host and each other exactly the way I want them to be, managed from one location.

The issue is (according to the bridge(4) man page) that "If the destination is on the same segment as the origin segment, the bridge will drop the packet because the receiver has already had a chance to see the frame."

Now this is my issue here, I'm having a difficult time getting a work around together (that I'm assuming involves removing the interface in question from the bridge), mostly because I'm not sure what components of OpenBSD (like a !route line in hostname.if) are operating in the layer that I'm trying to deal with (layer2?). I have tried to disable stp on the port in question (using a line in the hostname.if), I have tried bridging vether devices together so I can merry-go-round the frames back to the interface, but the bridges won't seem to allow a vether device on more than one bridge. I have also tried integrating trunks and vlans, to no success.

The biggest thing that has brought me to the forum with this issue is that my experiments with the various psudo-devices are not doing what I am predicting they will in any way. I am guessing this is a result of the psudo-devices being designed to work concisely, and not in the way that I'm using them.

Linux has a 'Hairpin mode' and some physical switches offer ‘Reflective Relay’ mode, I had assumed that I could implement this in OpenBSD.

If someone could describe to me a theory that I could try to implement, I would me most appreciative.

https://seravo.fi/2012/virtualized-b...g-with-macvtap

Thank you,

Mike
__________________
The bugs will pass, but the functionality will remain.

Last edited by rbigm101; 13th September 2016 at 09:33 PM. Reason: Title lacked clarity, lacked example of this functionality in other systems, lacked my OpenBSD version
Reply With Quote
  #2   (View Single Post)  
Old 13th September 2016
ocicat ocicat is offline
Administrator
 
Join Date: Apr 2008
Posts: 3,277
Default

Quote:
Originally Posted by rbigm101 View Post
I'm using OpenBSD 5.0 i386
Are you using OpenBSD 5.0 or 6.0?
Reply With Quote
  #3   (View Single Post)  
Old 13th September 2016
jggimi's Avatar
jggimi jggimi is online now
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 5,981
Default

Hello and welcome!

Is there a way to meet your topology requirements without using bridges? An external managed switch, for example? Or perhaps vxlan(4) will do what you need. The vxlan driver was designed to tunnel complete virtual networks over cloud services, and was added with OpenBSD release 5.5.
Reply With Quote
  #4   (View Single Post)  
Old 14th September 2016
rbigm101 rbigm101 is offline
New User
 
Join Date: Sep 2016
Posts: 8
Default

Quote:
Originally Posted by ocicat View Post
Are you using OpenBSD 5.0 or 6.0?
I'm using OpenBSD 5.0
__________________
The bugs will pass, but the functionality will remain.
Reply With Quote
  #5   (View Single Post)  
Old 14th September 2016
rbigm101 rbigm101 is offline
New User
 
Join Date: Sep 2016
Posts: 8
Default

Quote:
Originally Posted by jggimi View Post
Hello and welcome!

Is there a way to meet your topology requirements without using bridges? An external managed switch, for example? Or perhaps vxlan(4) will do what you need. The vxlan driver was designed to tunnel complete virtual networks over cloud services, and was added with OpenBSD release 5.5.
I believe I need to use bridges, as the switch will allow the VM's to talk to each other unless I implement a second firewall (which I am trying to avoid), and I don't know of any other way to connect my interfaces besides a bridge, which are as follows:
I have a WAP on one interface, my Debian server with the VM's on another interface, my Desktop on a third interface, and the internet on a fourth interface. The WAP is running dd-wrt.

I will look into the vxlan possibility. Which means I will be upgrading to OpenBSD 6.0, and that may take me a few days. Thanks you for the information. I'll update the thread with my results.
__________________
The bugs will pass, but the functionality will remain.

Last edited by rbigm101; 14th September 2016 at 02:29 AM. Reason: Provided reason for needing bridge, and details on my network topology.
Reply With Quote
  #6   (View Single Post)  
Old 14th September 2016
jggimi's Avatar
jggimi jggimi is online now
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 5,981
Default

See the vxlan(4) man page, and then this presentation.
Reply With Quote
  #7   (View Single Post)  
Old 14th September 2016
jggimi's Avatar
jggimi jggimi is online now
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 5,981
Default

I should add, that's a FreeBSD presentation but it gives a detailed explanation of the tunneling mechanism. I looked for an OpenBSD-specific presentation, as I recalled seeing one, but perhaps I am thinking of the video of reyk@'s presentation in 2014.
Reply With Quote
  #8   (View Single Post)  
Old 14th September 2016
ocicat ocicat is offline
Administrator
 
Join Date: Apr 2008
Posts: 3,277
Default

Quote:
Originally Posted by rbigm101 View Post
I'm using OpenBSD 5.0
OpenBSD 5.0 was released in November 2011, & support ended in November 2012.
Reply With Quote
  #9   (View Single Post)  
Old 14th September 2016
rbigm101 rbigm101 is offline
New User
 
Join Date: Sep 2016
Posts: 8
Default

Quote:
Originally Posted by ocicat View Post
OpenBSD 5.0 was released in November 2011, & support ended in November 2012.
I'll update the thread after I've updated to 6.0. It'll take me a few days.
__________________
The bugs will pass, but the functionality will remain.
Reply With Quote
Old 18th September 2016
rbigm101 rbigm101 is offline
New User
 
Join Date: Sep 2016
Posts: 8
Default

Alright so I'm up and running with OpenBSD 6.0. Right now I have my interfaces bridged, with DCHP running off of a vether interface.

My thoughts in the meantime on getting my VEPA working have been as follows:

1. I can't think of a way to attach the vxlan to my interface (without a bridge) in a way that seems like it would work.

2. I could remove the interface to the KVM host from the bridge, and do some kind of "rdr-to and nat-to combination" as outlined here, http://www.openbsd.org/faq/pf/rdr.html#rdrnat , but this is going to involve getting this to work, while also restoring "bridge like" functionality.

So my question to the forum now is, is #2 my best option, and what is the general idea behind how this would be implemented? I'm finding a working theory very challenging to develop.
__________________
The bugs will pass, but the functionality will remain.
Reply With Quote
Old 18th September 2016
LeFrettchen's Avatar
LeFrettchen LeFrettchen is offline
Marveled user
 
Join Date: Aug 2012
Location: Bronvaux, France
Posts: 264
Default

Quote:
Originally Posted by ocicat View Post
OpenBSD 5.0 was released in November 2011, & support ended in November 2012.
Who cares ?
I still use OpenBSD 5.1 on my old laptop (Pentium 3), and it rocks.
__________________
The secret of wisdom is not wisdom itself, it's the road that leads us there.
Reply With Quote
Old 18th September 2016
jggimi's Avatar
jggimi jggimi is online now
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 5,981
Default

Quote:
Originally Posted by rbigm101 View Post
I can't think of a way to attach the vxlan to my interface (without a bridge) in a way that seems like it would work.
Keep in mind, vxlan(4) must be used by all participating VMs. These tunnels might be able to "bridge" using multicasting, as described in section 2.4 of Reyk's paper, which begins on page 91 of the AsiaBSDCon 2014 proceedings. https://2016.asiabsdcon.org/papers/abc2014-proc-all.pdf

Quote:
Originally Posted by LeFrettchen View Post
Who cares ?
I still use OpenBSD 5.1 on my old laptop (Pentium 3), and it rocks.
In this particular case, rocking doesn't matter. The vxlan(8) driver was not available until 5.5.

As for your 5.1-release laptop, there have been 71 critical security and reliability fixes published since it was released. Of these 71, only 1 is available for 5.1. Should you care? Well, that's entirely up to you.
Reply With Quote
Old 18th September 2016
TronDD TronDD is offline
Package Pilot
 
Join Date: Sep 2014
Posts: 155
Default

Quote:
Originally Posted by LeFrettchen View Post
Who cares ?
Maybe the people who volunteer their time to provide support.

Run whatever you want but time moves on. If you didn't keep up, you can't expect many people remember how the system worked 5 years ago, or to have a setup where they can help investigate your problem.
Reply With Quote
Old 18th September 2016
rbigm101 rbigm101 is offline
New User
 
Join Date: Sep 2016
Posts: 8
Default

Quote:
Originally Posted by jggimi View Post
Keep in mind, vxlan(4) must be used by all participating VMs. These tunnels might be able to "bridge" using multicasting, as described in section 2.4 of Reyk's paper, which begins on page 91 of the AsiaBSDCon 2014 proceedings. https://2016.asiabsdcon.org/papers/abc2014-proc-all.pdf)
I hadn't considered that I might be able to connect the xvlan(4) directly to the VM (or any other type of implementation that did not include using bridge(4)), as the example implementation of xvlan(4) that I found to help me understand what xvlan(4) fundamentally was, seems to show that is needs to be a part of a bridge(4) to be useful. That refrence is here: http://www.echothrust.com/blogs/usin...ay-remote-lans

I'll look through that paper to see if I can discover an implementation of xvlan(4) that will relate to my situation. Thank you for your reply.
__________________
The bugs will pass, but the functionality will remain.
Reply With Quote
Old 18th September 2016
jggimi's Avatar
jggimi jggimi is online now
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 5,981
Default

Quote:
Originally Posted by rbigm101 View Post
I'll look through that paper to see if I can discover an implementation of xvlan(4) that will relate to my situation.
His video (link above) may be helpful, and if you need to, you might consider reaching out to him directly.
Reply With Quote
Old 20th September 2016
rbigm101 rbigm101 is offline
New User
 
Join Date: Sep 2016
Posts: 8
Default

Update:
I have been configuring my VM’s to use the vxlan. I have the vxlan that is attached to the VM’s bridged to my eth0 interface. I have the other vxlan on my OpenBSD router, on the same bridge as all the interfaces. The purpose of this experiment was to gain/demonstrate the functionality of having one of the VM’s talk to its host system. The VM talkes to to xvlan, the xvlan repeats on the OpenBSD bridge, the bridge sends the frame to the KVM host ethernet port.

Here’s what keeps happening:
1. I have my OpenBSD router running with the vxlan ready to go.

2. I turn on my KVM host and configure the vxlan with these commands.

Code:
ip link add vxlan0 type vxlan id 1 remote 172.168.1.1 local 172.168.1.11 dev eth0 dstport 4789
ip addr add 172.168.1.200/24 dev vxlan0
Then I start my VM’s

3. I run tests with ping, telnet, and tcpdump, while using X11 forwarding to view/use the running VM’s, so I can do ping, and telnet tests on those, while watching my tcpdump.

4. At some point in my testing, the OpenBSD kernel will crash. I have noticed that these kernel crashes will not reoccur if I restart the OpenBSD router after a kernel crash and continue testing, without having restarted my KVM host along with it. Meaning I just ssh into the KVM host again and continue my testing. If I restart the KVM host, shortly after I start testing, I'll do a ping or try to ssh and the router will crash. Photos of all of the information I could produce from the crashes are below.

I have not been able to get the xvlan to work at any point durring this (not even an ip assignment to the VM’s) , and I’m worried that the functionality I’m looking for is not available while the router is the last thing that has been booted. Although it could be just poor vxlan configuration and/or firewall issues.

Tomorrow I’m going to submit a bug report on the kernel crashes, and a bug report on the bridge(4) for not supporting a “hairpin” mode, which I believe is the most elegant solution to using VEPA with OpenBSD. I’m also going to look into different ways to connect the vxlan to the VM’s.

http://s12.postimg.org/x2vfmecf1/IMG...003886_HDR.jpg
http://s12.postimg.org/rsqgv3s65/IMG...547944_HDR.jpg
http://s12.postimg.org/ke152q6al/IMG...023029_HDR.jpg
http://s12.postimg.org/riiybrdjx/IMG..._185802405.jpg
__________________
The bugs will pass, but the functionality will remain.

Last edited by rbigm101; 20th September 2016 at 12:12 AM. Reason: Forgot to explicitly say that when I restarted my KVM host, it crashes. Forgot to blame firewall/self for non functionality.
Reply With Quote
Old 20th September 2016
jggimi's Avatar
jggimi jggimi is online now
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 5,981
Default

Quote:
Originally Posted by rbigm101 View Post
...a bug report on the bridge(4) for not supporting a “hairpin” mode...
A lack of a feature is not considered a bug, even if you determine the feature is required for your particular implementation.
Reply With Quote
Old 20th September 2016
rbigm101 rbigm101 is offline
New User
 
Join Date: Sep 2016
Posts: 8
Default

Quote:
Originally Posted by jggimi View Post
A lack of a feature is not considered a bug, even if you determine the feature is required for your particular implementation.
Then I'll title it feature request, I've seen feature requests in bug reports before.

Also, I had left the vxlan configured on my KVM host overnight, after the last kernel crash. This morning I saw that it had crashed again (pic below), so the "KVM post OpenBSD crash reboot theory" is out.

I need to move on with my VM based project, so I'm going to try and make an OpenBSD firewall/switch VM that is bridged to my physical router with vxlan. The VM's will be on an isolated network. I hope this will be possible without havingto put the VM's on a different subnet. I may try to add the functionality that I need to bridge(4) myself in the near future.

http://s17.postimg.org/homs04inj/IMG..._112525508.jpg
__________________
The bugs will pass, but the functionality will remain.

Last edited by rbigm101; 20th September 2016 at 05:47 PM.
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Building a Firewall/Router prepurchase questions azarian OpenBSD Security 19 16th January 2015 11:05 AM
pf firewall, is it a bridge or router? tomp OpenBSD Security 8 17th August 2011 06:12 PM
OpenBSD amd64 or i386 for firewall/router J65nko OpenBSD General 7 24th December 2009 09:06 PM
Is there a purpose for using pf if you have a hardware router/firewall? guitarscn OpenBSD Security 9 23rd January 2009 12:22 AM
FreeBSD as firewall/router on VMware ESXi Bruco FreeBSD General 12 6th December 2008 08:37 PM


All times are GMT. The time now is 01:01 PM.


Powered by vBulletin® Version 3.8.4
Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.
Content copyright © 2007-2010, the authors
Daemon image copyright ©1988, Marshall Kirk McKusick