Network benchmarks
As a concept, I understand why you would want to test your NICs in this manner. With a cross-over cable there are no external devices that might inadvertantly skew performance results.
However... even if you
could get a functional test, your results will not match any real network load. And, unles your test is able to peak CPU 0 at 100% load you will be unable to isolate the component that limits performance. (With this OS, the network stack is in-kernal and runs only on CPU 0.) And on OpenBSD, there's no way to set affinity to other CPUs for your tcpbench processes.
I believe you should test with at least three systems: the client, the server, and the system under test.
When benchmarking network performance, you should model your actual network. I will refer you to a recent discussion thread on the misc@ mailing list about 10Gbit Ethernet benchmarks. The discussion thread began
here. You might find it helpful.
Two NICs on one subnet
You need to be very careful if you do this. As I mentioned above, I never recommend it. If you search (perhaps with Google) using the search terms
two nics one subnet, you will find many recommendations to avoid this. You can have problems with ARP resolution and routing on the subnet. I will refer you to the first hit on Google with those search terms in that order, a brief white paper from National Instruments which may be helpful:
Best Practices for Using Multiple Network Interfaces (NICs) with NI Products
Loopback routing via two NICs
Even if you are able to correctly configure two Ethernet NICs on one subnet, you cannot use a single layer 2 IP subnet in the manner you intend, due to how subnets on IP are defined and routed.
You might be able to establish and use a point-to-point tunnel on separate subnets, using
gif(4) or
gre(4), but then you would be testing the tunnel protocol's performance over Ethernet, rather than Ethernet, and you would still not be sure you were connecting via
lo(4) or your NICs without further analysis.