OpenBSD amd64 or i386 for firewall/router

[J65nko]

In a discussion about OpenBSD network performance on an Atom motherboard http://www.daemonforums.org/showthread.php?t=4090
a question arose wich of the OpenBSD architectures, amd64 or i386, would be best for a firewall or router.

I finally managed to find the posts in my gmail account archive. And being in a lazy mood today I only give you the start of this long discussion on the misc mailing list.

From Henning Brauer
to misc
date Tue, Oct 9, 2007 at 12:32 PM
subject Re: firewall is very slow, something's wrong


* Florin Andrei <florin@xxxxxx> [2007-10-05 03:55]:
> The hardware is AMD64, Tyan Transport, 2 CPUs 2 cores each. I am using the
> SMP kernel. The network card is Intel Pro/1000 PCI Express 4x dual gigabit
> port, it carries both em0 and em1.

First, you want to run 4.2 or -current, that shoudl about double your
throughput.
then, an i386 kernel should perform considerably better than amd64 for
firewalling/routing/...

next, you don't want SMP for such tasks. take out the second CPU and
give it to somebody who can use it, and run the uniprocessor kernel.

last, increase net.inet.ip.ifq.maxlen until you see the congestion
counter not increasing much any more under load. should not exceed 2500
by too much. as a rule of thumb, 256 per gigE interface aren't too far
off.

In the same thread: he kind of relativizes this:

from Henning Brauer
to misc
date Tue, Oct 9, 2007 at 7:03 PM
subject Re: firewall is very slow, something's wrong

Florin Andrei <xxx> [2007-10-09 19:34]:
>> then, an i386 kernel should perform considerably better than amd64 for
>> firewalling/routing/...
>
> That is surprising. What is the reason?

we dunno really. it hasn't been benched in sometimesoit might not even
be true nay more, but last time the difference was dramatic.

The last post of Henning in this thread he still recommends i386 for a router.

From Henning Brauer
to misc
date Wed, Oct 10, 2007 at 9:20 PM
subject Re: firewall is very slow, something's wrong


* Robert C Wittig [2007-10-10 20:45]:
> If you had to choose between, say, 2 gig RAM and a 32 bit CPU, or 1 gig RAM
> and a 64 bit CPU, which would be a better choice, in general?

for a packet filter/router/...? 32bit 2Gig and take a gig out.
for a databse server? 64bit and add ram when required.
there is no "in general".

The thread has many interesting posts , e.g. "does a 20 ton truck run faster then a 10 ton truck?", where the 20 ton truck stands for amd64 and the 10 ton one for i386.

[BSDfan666]

I respect Henning's work on PF, and without having a greater understanding of the differences between 64-bit and 32-bit platforms.. I will not contest this.

Sorry J65nko, I wasn't intending to second guess anything.. it just seemed unbelievable to me.

I never considered AMD64 a "20 ton truck", I thought the added benefits of being able to do 64-bit arithmetic easily would be like a 10 truck with hover conversion.. processing double the load in the same amount of time.

[J65nko]

You are not the only one, I was rather surprised too.

I was running OBSD amd64 on a board that I got as a present for my birthday in July of that year. Quite a leap from the 800MHz PIII I used before. Three months later Henning tells me amd64 possibly could be not the ultimate version.

But if you just imagine linked lists with 64 bit pointers in an amd64 version, you can imagine that traversing such a list takes more time then the same machine using 32 bit pointers for a similar linked list structure in i386 mode.
Henning mentioned something like this in another thread, but I haven't found it yet.

BTW In the same thread somebody points out the role of the gcc compiler. IIRC the quality of the 64 bits generated code hasn't caught up with the i386 in all cases.

Well at least we know for a database server, amd64 and a lots of RAM is the recommended platform.

[Oliver_H]

>IIRC the quality of the 64 bits generated code hasn't caught up with the i386 in all cases.

In common or regarding OpenBSD? They are using yet gcc2.95 as far as I know? I'm using 64bit especially for 4G of memory, so it's a must.

[J65nko]

For the compiler issue see http://www.mail-archive.com/misc@ope.../msg49069.html

Also see http://en.wikipedia.org/wiki/GNU_Com...e_alternatives

OpenBSD doesn't use the latest and greatest gcc, because gcc developers, mainly from a couple of Linux companies like RedHat, have dropped support for some hardware platforms.
Why should a company like Redhat sponsor gcc development for hardware which are for no commercial value for Redhat?

If OpenBSD still wants to support these for gcc obselete architectures, it has no other choice then using an older gcc version.

IIRC NetBSD is dealing with this unfavourable condition by using several gcc versions.

[BSDfan666]

Originally Posted by Oliver_H

>IIRC the quality of the 64 bits generated code hasn't caught up with the i386 in all cases.

In common or regarding OpenBSD? They are using yet gcc2.95 as far as I know? I'm using 64bit especially for 4G of memory, so it's a must.

There are 3 versions of GCC in the tree, gcc2/gcc3 and as of late.. gcc4.

Both amd64 and i386 use a modified (..gcc-local(1)) version of gcc-3.3.5, a few older architectures use gcc-2.95.3.

In the future, some architectures will be switched over to gcc-4.2.1.. and then hopefully pcc.

GCC 4.2.1 was the last version released under the GPLv2, newer versions are GPLv3.

Here is the commit log for it:
http://marc.info/?l=openbsd-cvs&m=125562729215306&w=2

[Carpetsmoker]

One advantage of 64 bit OpenBSD is the availability of NX bit...

Is OpenBSD now able to use more than 4GB of memory?

[BSDfan666]

No not yet, is a unsupported toggle in the code for it.. but multiple developers have stated it's broken for now.

People using amd64 still have the access to a larger virtual address space, although I think it's set to around 8GB or so per process.. for now anyway.

As for the NX bit, OpenBSD has always simulated that on most architectures via W^X.. the introduction of the NX bit on AMD64 just made it simpler.

AFAIK, newer CPU's supporting the NX bit can also make use of it on the i386 platform, I could be wrong though.. they use a different (..slightly more costly) alternative on processors that do not, using a segmentation trick.

A paper by Theo: http://www.openbsd.org/papers/ven05-deraadt/index.html