FreeBSD as firewall/router on VMware ESXi

[Bruco]

I'm going to be inheriting an old Compaq Proliant server from work to use at home. It's nothing special, has a couple PIII procs, 4GB of RAM. But it will run VMware's free ESXi product on it, which I'd like to do. We're moving to VMware at work and the more experience I get, the better. Besides, it's fun to play with!

One thing I'd like to consider doing is running a FreeBSD virtual box as my router/firewall for my home network (I have DSL). Right now that job is handled by a Buffalo router with DD-WRT on it, and I've learned some iptables from it, but I'd really like to learn from the ground up with pf.

However, my concern is the miserable network performance I think I'm seeing from FreeBSD 6.3 as a virtual device (I had problems loading 7.0 on ESXi). I read this page:

http://taosecurity.blogspot.com/2007...nterfaces.html

and it certainly makes sense, with the lnc driver network performance was pretty sad. Even pinging localhost gives me times around .165 ms, where as on two physical FreeBSD boxes (using fxp and sis network drivers) pinging localhost gives me responses in less than half that time.

I did use the article's advice and changed to the em driver. That did cut my ping to localhost time down a little bit (more like .141 ms). But obviously that still isn't as good as a physical box.

So, my questions would be, to anyone that has experience with this:

1. Is it worth my time to either recompile the 6.3 kernel or fight with 7.0 until it installs so I can use the le network driver instead of lnc or em?
2. Is it foolish to even attempt using a virtual machine as a gateway and firewall? (Obviously this is would just be for a tiny home network on a DSL line, so maybe I wouldn't notice a difference, but I'm curious what people's opinions are.)

Thanks!

[J65nko]

I wouldn't use such an electrical power consuming beast for a firewall. IMHO an ecological disaster
Why don't you just get an old Pentium I or II with 2 network cards, or a small machine like a Soekris or Via ITX.

I think running a virtual machine as a firewall is a bad idea. Programmers already have a hard time writing secure operating systems. You think that in their initial attempt in writing secure virtualization layers they suddenly do better? I don't

[Bruco]

I'd love a low-power solution, because I really want to be able to play with ESXi at home and learn more about it. But there aren't any low-power servers they are giving away for free at work! Maybe I should just look into one of these websites devoted to whitebox hardware that works with ESXi. Of course, then I have to spend money!

Thanks for your input - security is a good concern. I'm still very curious about the latency of network I/O, too.

[tingo]

Originally Posted by J65nko

I think running a virtual machine as a firewall is a bad idea. Programmers already have a hard time writing secure operating systems. You think that in their initial attempt in writing secure virtualization layers they suddenly do better? I don't

Well, you are free to do some research and post any definite findings that indicate the this virtualization product (or any other virtualization product on the Intel PC platform) is less secure than running on a physical server. After all, vmware have been running for a few years now. And the Hypervisor concept and first implementation is now ancient. :-)

[Oko]

Originally Posted by tingo

Well, you are free to do some research and post any definite findings that indicate the this virtualization product (or any other virtualization product on the Intel PC platform) is less secure than running on a physical server. After all, vmware have been running for a few years now. And the Hypervisor concept and first implementation is now ancient. :-)

This is my last post on the topic as I do not want to get into any kind of flame wars. People will do what
they want to do no matter what. If you want to run firewalls on the top of WM it is your choice.
It might be a good choice if you wan balance your desired level of security and cost of deployment.
A real good solution might be cost prohibitive.

Now back to my answer.
Let me para-phrase Theo De Raadt:"Running your crap on the top of somebody else crap is not going to make your crap any better". Any peace of code (even couple lines) is prone to bugs. Adding another layer between
your crappy OS (yes even OpenBSD is crap as well just far less than other Operating System) and a crappy PC machine (if nothing else sparc64 is at least less prone to buffer overflow) is not going to make you safer.

If OpenBSD which is debugged for more than 10 years now still have bugs
I can just imagine WMWare. When OpenBSD was firstly ported to Motorola 88000 new bugs were discovered on weekly basis. Some of them
were in BSD Unix for more than 20 years. I am sure WMware team have ported WMware on 10 different processor architectures like OpenBSD just to find the bugs. So how long did you run WMware on Motorola 68000 or 88 000 processors. How long have you run it on SGI Mips architecture?
Does it run flawlessly?

[windependence]

Originally Posted by Oko

This is my last post on the topic as I do not want to get into any kind of flame wars. People will do what
they want to do no matter what. If you want to run firewalls on the top of WM it is your choice.
It might be a good choice if you wan balance your desired level of security and cost of deployment.
A real good solution might be cost prohibitive.

Now back to my answer.
Let me para-phrase Theo De Raadt:"Running your crap on the top of somebody else crap is not going to make your crap any better". Any peace of code (even couple lines) is prone to bugs. Adding another layer between
your crappy OS (yes even OpenBSD is crap as well just far less than other Operating System) and a crappy PC machine (if nothing else sparc64 is at least less prone to buffer overflow) is not going to make you safer.

If OpenBSD which is debugged for more than 10 years now still have bugs
I can just imagine WMWare. When OpenBSD was firstly ported to Motorola 88000 new bugs were discovered on weekly basis. Some of them
were in BSD Unix for more than 20 years. I am sure WMware team have ported WMware on 10 different processor architectures like OpenBSD just to find the bugs. So how long did you run WMware on Motorola 68000 or 88 000 processors. How long have you run it on SGI Mips architecture?
Does it run flawlessly?

This makes me upset when I read it. Do you think that the banks, insurance companies, etc who run VMware are not concerned about security? (and there are a lot of them)

Read here

and here .

This is a clip from the VMware site:

Strengthen Security & Improve Reliability

VMware ESXi is the only hypervisor that does not incorporate or rely on a general-purpose operating system (OS), eliminating many common reliability issues and security vulnerabilities. The slim 32MB footprint of VMware ESXi is a fraction of the size of a general-purpose operating system, resulting in a smaller attack surface while minimizing the effort required for tasks such as security hardening, user access control, anti-virus and backup. In addition, integration into solid-state components inside a server enables diskless server configurations. This reduces hardware failure rates and decreases server power consumption.

Indeed, the small footprint makes it less of a target. I am not saying it's perfect. Even one of my favorite OSes, OpenBSD is not perfect, but I have several VM firewall implementations in production environments and I have found it actually easier to secure them due to the fact that I can create virtual networking to isolate traffic on different networks. I just installed a box very similar to the one the OP is thinking about using (it's a dual PIII Dell 2550). We run 2 pfsense firewalls on it, one for the general network and one for the mail server. I keep all the mail server traffic on a completely separate and different network so that if there is a virus outbreak on the LAN, the e-mail server is not affected and vice versa.

Of course, someone will always be able to take advantage of exploits if there is any, but considering that the DOD has certified Windoze, this is order of magnitude better than that situation. ESXI can also be run from a CF card or a thumb drive. We are going to start using dual CF cards and use the disks only for the VMS. That should be even more secure.

Finally, the Dell 2550 we put in didn't draw THAT much power, even with 5 SCSI drives. Of course, I'm not that politically correct either.

One thing I forgot to mention. SATA drives are not supported for VM storage unless you use certain controllers. Be careful. I built a $2,000 box only to find out I had to put the 2TB of storage in the SAN.

-Tim

[J65nko]

You still can play with that server and with ESXi. I just wouldn't use it as a firewall

If you use another box that thas 3 NIC's you can place your server in a DMZ.
NIC one is the external interface, NIC two the internal interface. The last NIC is used for the DMZ.
Because you have a single server machine in the DMZ you don't need a switch to connect it to the DMZ interface. Just a simple cross-over UTP cable.

Using VMware you then can configure a nameserver, ftp server, www server etc.

[Yuka]

I find it odd that you are having network performance issues with VMWare ESXi + FreeBSD.

I've had excellent luck with it, both VMWare ESXi and VMWare Server (windows as the Host OS). With VMWare server on windows I saw 5 to 8 MB/sec from a FreeBSD 7.0 webserver and on VMWare ESXi, I saw it max out the theoretical bandwidth after protocal overhead on the NICs that were available. (Same FreeBSD 7.0 webserver.)

If there's anything I can do to help compare your system to mine, please let it be known. (I'm pretty new to FreeBSD and 'nix in general but I'm willing to help as much as I can.)

Just watch out for the time issue, it's nasty!

As for security as a border firewall/router... I can't speak for it in either way as I've never dabbled with a virtual guest as such. I can see a few cases where it would be feasible, but you'd need some pretty bulletproof VLANs and other nasties.

[Bruco]

Originally Posted by Yuka

I find it odd that you are having network performance issues with VMWare ESXi + FreeBSD.

I've had excellent luck with it, both VMWare ESXi and VMWare Server (windows as the Host OS). With VMWare server on windows I saw 5 to 8 MB/sec from a FreeBSD 7.0 webserver and on VMWare ESXi, I saw it max out the theoretical bandwidth after protocal overhead on the NICs that were available. (Same FreeBSD 7.0 webserver.)

I think my concern is less about throughput than latency. I don't need any extra lag from my router when I'm playing some Team Fortress 2, after all!

I've decided I probably won't lug home one of these old dual-PIII servers - the power consumption would probably make me regret it! What I will do is look at putting together a whitebox than can run ESXi - I really like it as a test environment.

In the meantime, I have an old Dell GX240 that I can try as a pf box. Maybe once I get an ESXi box I can replicate the setup onto a virtual machine and compare network performance.

[Oko]

Originally Posted by Bruco

2. Is it foolish to even attempt using a virtual machine as a gateway and firewall? (Obviously this is would just be for a tiny home network on a DSL line, so maybe I wouldn't notice a difference, but I'm curious what people's opinions are.)

I think it is ridiculus but I am just a hobbyist as my Linux friends say so I am
clueless

[tingo]

Originally Posted by Bruco

2. Is it foolish to even attempt using a virtual machine as a gateway and firewall? (Obviously this is would just be for a tiny home network on a DSL line, so maybe I wouldn't notice a difference, but I'm curious what people's opinions are.)

No, it is not foolish - this is a perfectly acceptable solution for many cases.
Especially if you run a lot of virtual servers, then running a virtual firewall (or a virtual firewall cluster) makes sense.

[J65nko]

Windependence, please never read the thread http://marc.info/?l=openbsd-misc&m=119304079603049&w=2, that will make you more upset

[tingo]

Here is a paper in favor of virtualization:
http://shell.cse.ucdavis.edu/~bill/virt/virt.pdf
And this post addresses some of the Xen issues in Ormandys paper:
http://marc.info/?l=openbsd-misc&m=119324474519679&w=2