DaemonForums  

Go Back   DaemonForums > OpenBSD > OpenBSD General

OpenBSD General Other questions regarding OpenBSD which do not fit in any of the categories below.

Reply
 
Thread Tools Display Modes
  #1   (View Single Post)  
Old 10th January 2013
xinform3n xinform3n is offline
Port Guard
 
Join Date: Jun 2009
Posts: 15
Thanked 0 Times in 0 Posts
Arrow Link aggregation performance

Hello,

I'm a little bit disappointed with network performance of OpenBSD and I'm searching other point of view.

background
I need to put traffic load on a network lab. Naturally, I thought of my favorite distribution,OpenBSD.

So I've installed two HP ML370G3 server with OpenBSD 5.2 i386.
Each server is equipped with two Intel dual NIC gigabit (plus one embedded gigabit NIC), two Xeon 3.2GHz H.T. and 12GB RAM.

I have installed two tools: iPerf and NetPerf.
I did this following experiment with this two tools, but the results are similar.

From network side, the topology is as follows:
A core network with two big chassis and two high performance access stacks (switchs).
Each access stack is connected with two 10Gbps fiber links in aggregation.

experiment 1
Server A - NIC em0 - Subnet A - @.111
Server B - NIC em0 - Subnet B - @.222

One-way test (64KB window): ~ 870Mbps

experiment 2
Server A - NIC trunk0 - Subnet A - @.111
Server A - NIC em0 - trunkport, loadbalance
Server A - NIC em1 - trunkport, loadbalance

Server B - NIC trunk0 - Subnet B - @.222
Server B - NIC em0 - trunkport, loadbalance
Server B - NIC em0 - trunkport, loadbalance

Switch side is well configured in aggregation with a hash algorithm based on src.mac+dst.mac+src.ip+dst.ip+src.port+dst.port

One-way test (64KB window): ~ 870Mbps
Two parallel one-way test (64KB window): total don't exceed ~ 850Mbps

experiment 3
Server A - NIC trunk0 - Subnet A - @.111 and @.112
Server A - NIC em0 - trunkport, loadbalance
Server A - NIC em1 - trunkport, loadbalance

Server B - NIC trunk0 - Subnet B - @.222 and @.223
Server B - NIC em0 - trunkport, loadbalance
Server B - NIC em0 - trunkport, loadbalance

Switch side is well configured in aggregation with a hash algorithm based on src.mac+dst.mac+src.ip+dst.ip+src.port+dst.port

One-way test (64KB window): ~ 870Mbps
Two parallel one-way test (64KB window, on different IP @): total don't exceed ~ 850Mbps

experiment 4
Server A - NIC trunk0 - Subnet A - @.111 and @.112
Server A - NIC em0 - trunkport, lacp
Server A - NIC em1 - trunkport, lacp

Server B - NIC trunk0 - Subnet B - @.222 and @.223
Server B - NIC em0 - trunkport, lacp
Server B - NIC em0 - trunkport, lacp

Switch side is well configured in aggregation with LACP standard.

One-way test (64KB window): ~ 870Mbps
Two parallel one-way test (64KB window, on different IP @): total don't exceed ~ 850Mbps

experiment 5
Server A - NIC em0 - Subnet A - @.111
Server A - NIC em1 - Subnet B - @.111

Server B - NIC em0 - Subnet A - @.222
Server B - NIC em0 - Subnet B - @.222

Two parallel one-way test (64KB window, on different IP @): total don't exceed ~ 850Mbps

question

Why am I blocked at ~ 1Gbps limit ?
  • Trunk driver ?
  • Kernel performance ?
  • PCI-X Bus limitation ?
  • CPU or FSB limitation ?
  • ... ?

Thanks for your help.
Reply With Quote
  #2   (View Single Post)  
Old 10th January 2013
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 3,675
Thanked 214 Times in 189 Posts
Default

I'm not sure how you were benchmarking. To my understanding, "loadbalance" will not aggregate a single communication session. The aggregation hash used is a combination of MAC addresses, IP addresses, and (if used) the VLAN tag. If those don't change, that may be why you may not see any gain.

Last edited by jggimi; 10th January 2013 at 02:03 PM. Reason: typo, clarity
Reply With Quote
  #3   (View Single Post)  
Old 10th January 2013
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 3,675
Thanked 214 Times in 189 Posts
Default

Here's a recent thread on misc@ about 10gbit performance tests...

http://marc.info/?l=openbsd-misc&m=134313556730356&w=2
Reply With Quote
  #4   (View Single Post)  
Old 10th January 2013
xinform3n xinform3n is offline
Port Guard
 
Join Date: Jun 2009
Posts: 15
Thanked 0 Times in 0 Posts
Smile

Quote:
Originally Posted by jggimi View Post
... The aggregation hash used is a combination of MAC addresses, IP addresses, and (if used) the VLAN tag. ...
Yes, that's correct. The man page says:

Quote:
Distributes outgoing traffic through all active ports and accepts incoming traffic from any active port.
A hash of the protocol header is used to maintain packet ordering.
The hash includes the Ethernet source and destination address, and, if available, the VLAN tag, and the IP source and destination address.
That's why in my third experiment I tried with two IP addresses per server (alias), but without more success.

The strangest part and the most disturbing for me is the last experiment.
Two independent NIC can't break ~ 1Gbps.
So I think that the root cause isn't the trunk driver.

I forgot to mention that PF was disabled. In this case I'm only interested by RAW performance.

Thank you jggimi the thread is very interesting. In summary:
  • There is a problem with OpenBSD.
  • They don't mention a solution.
  • The Calomel.org tuning isn't recommended.

I have also tested the Calomel.org tuning without any measurable performance enhancement. The threads say's:
Quote:
http://calomel.org/network_performance.html
Revert all changes recommended by this page, they hurt performance and do not help.
calomel.org is only bad juju.
Finally, where could be the bottleneck
Reply With Quote
  #5   (View Single Post)  
Old 10th January 2013
ocicat ocicat is offline
Administrator
 
Join Date: Apr 2008
Posts: 2,880
Thanked 190 Times in 160 Posts
Default

Quote:
Originally Posted by xinform3n View Post
Finally, where could be the bottleneck
The size of the community on this site is quite small, & the number of people who actually attempt to answer questions is smaller still. Given the specificity of your questions, my recommendation would be to post to the misc@ mailing list as that venue is where the project's developers reside, however as was seen in the earlier thread cited, issues do remain when it comes to performance.
Reply With Quote
  #6   (View Single Post)  
Old 11th January 2013
xinform3n xinform3n is offline
Port Guard
 
Join Date: Jun 2009
Posts: 15
Thanked 0 Times in 0 Posts
Thumbs up

Hi Ocicat,

That's a realistic approach. I'll do that and post update about the conduct of this new quest.

Thank you to both of you for brain process time
Reply With Quote
  #7   (View Single Post)  
Old 6th February 2013
xinform3n xinform3n is offline
Port Guard
 
Join Date: Jun 2009
Posts: 15
Thanked 0 Times in 0 Posts
Arrow

Hi,

The openbsd-misc mailing wasn't helpfull.
http://marc.info/?t=135886701500009&r=1&w=2

The discussion on this forum was much more useful !

So, if somebody as a good idea...
Reply With Quote
Reply

Tags
aggregation, lacp, performance, poor, trunk

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
NetBSD on D-Link DIR-100/300/320 Lexus45 NetBSD General 3 1st February 2011 12:24 PM
PPPoE on D-link 2640 vigol FreeBSD General 1 26th September 2010 03:25 PM
zyd0: no link ........... sleeping kallistoteles OpenBSD Installation and Upgrading 3 25th June 2010 02:38 PM
D-link (DI-524) router c0mrade General software and network 3 26th January 2009 08:14 AM
kde .desktop file link doesn't act like a link when opening files caesius FreeBSD Ports and Packages 3 14th October 2008 07:35 AM


All times are GMT. The time now is 06:21 AM.


Powered by vBulletin® Version 3.8.4
Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content copyright © 2007-2010, the authors
Daemon image copyright ©1988, Marshall Kirk McKusick