[ previous ] [ next ] [ threads ]
 
 From:  "Tomas Florian" <tomas at florien dot ca>
 To:  <m0n0wall at lists dot m0n0 dot ch>
 Subject:  Throughput problems (10 Mbs+ speeds)
 Date:  Sun, 11 Sep 2005 18:01:20 -0600
Hello,

I'm having very strange bandwidth problems with monowall.  I'm trying to
deploy a net4801 with 1.2b09 with network interface polling on a network
with 12 Mbs sustained.  It serves about 60 server machines on a class C
subnet.

The network setup is as follows:

  [ISP]
    |
    |17 Mbs capped line (typical 1-12 Mbs sustained)
    |
[Lucent Router]
129.152.205.1
    |
    |Cross over cable
    |
129.152.205.2
[Monowall]
129.152.147.1
    |
    |Switches
    |
[Servers x60]

I've tested the setup at home substituting the 60 servers with a single PC
and Lucent Router with a second PC.  I was able to successfully pass 95 Mbs
through the monowall (I used a traffic generator) The CPU usage hardly moved
from 1%, ping 20 ms (no losses) - the new polling option did wonders.  

However, when as soon as I tried to reproduce the same scenario at the
datacenter all sort of trouble started to happen. 

@5  Mbs - CPU usage 5%, 20 ms (no losses)
@7  Mbs - CPU usage 5%, 30 ms (no losses)
@9  Mbs - CPU usage 5%, 70 ms (40% losses)
@14 Mbs - CPU usage 5%, ?? ms (100%? losses)

This doesn't complare AT ALL with the results I got at home:

@95 Mbs - CPU usage 1%, 20 ms (no losses)

The configuration is identical, I just unplugged the box and brought it over
to the datacenter. 

The only difference that I could think of is that at home the 95 Mbs was
purely generated by D-ITG traffic generator.  At the datacenter it was a
combination of real traffic 1 Mbs + 4 to 15 Mbs generated traffic.

D-ITG generator sent UDP packets on port 8889 of length 1024 bytes -  I used
the same traffic generator at the datacenter as I did at home PC to PC setup
(of course the other end was sitting at another datacenter so that I could
achieve 15+ Mbs - coulnd't do that from my DSL at home :-)

What could account for such a large discrepancy between the results I got
PC-PC and the real setup which was essentially
PC-Router-Internet-Router-Monowall-PC ?

The first thing I was wondering is if the other routers had something to do
with the difference.  So I unplugged the monowall from the picture and I was
able to pass the 17 Mbs through them no problem what so ever ... so
obviously monowall became the bottle neck somehow.

I was hoping that I could give the monowall community the honor of having
monowall route up to 17 Mbs on a daily basis (until now I've been having
nothing but luck with 2 Mbs DSL lines)... but time is running out and I'm
completely dumbfounded by the discrepancy between the home test and the real
world test so I'll probably have to try some other solution :-| ... which I
would like to avoid.

Another thing that I was thinking is that things like the complexity of
filtering rules would have a big impact on performance so I made sure that I
had the following options:
- All traffic allowed through all interfaces (firewall effectively turned
off)
- Outbound NAT checked on (both WAN and LAN use routable IPs so I don't need
NAT)
- Traffic Shaper definitely OFF
- Polling checked on
- The rest are defaults for net4801 image

I'm out of ideas and I'd appreciate any feedback I can get. Is someone else
running net4801 equivalent hardware at ~15 to 20 Mbs?  What's the max?  The
monowall web site says max is ~75 Mbs - but I got as much as 95 Mbs and as
little as 8 Mbs .. what kind of environment can handle 75 Mbs?  Why does it
make such a huge difference between the PC - PC setup and the real world
setup?

Thank you very much,
Tomas