[ previous ] [ next ] [ threads ]
 From:  Chris Buechler <cbuechler at gmail dot com>
 To:  Jim Gifford <baadpuppy at gmail dot com>
 Cc:  m0n0wall at lists dot m0n0 dot ch
 Subject:  Re: [m0n0wall] PIX vs m0n0wall testing -- requesting input (was: Re: [m0n0wall] Re: Re: Re: to Wrap or to Soekris that is my question? What about VIA?)
 Date:  Thu, 16 Dec 2004 11:51:27 -0500
On Wed, 15 Dec 2004 10:08:55 -0500, Jim Gifford <baadpuppy at gmail dot com> wrote:
> I have access to some PIX 506E models, and also some 515 models.  I
> also have a soekris 4801 I could use.  I would love to know how these
> all stack up against each other.  I could probably get my hands on a
> reasonable performing standard intel box as well.
> However, in the interest of doing fair testing, I would imagine it
> would be best to configure all devices as close to identically as
> possible.  Then, using the same sets of servers and clients do the
> throughput tests, just changing out the appliance between test runs.

I'd go with crossover cables only between devices (eliminate any
influence a switch might have), and try NAT and routing speeds

> I believe that we can do this at work.  My coworker is also interested
> in the results.  He is a believer in PIX and I'm a m0n0wall
> evangelist, however I believe together we can give good fair testing
> and results.

I just ran iperf from LAN to DMZ (routing, no NAT) on a 515E at a
steady 91-92 Mb.

Compared to Soekris and WRAP, that blows them out of the water, but we
aren't talking even close to the same class of hardware (that PIX
costs around 12 times as much as a Soekris or WRAP).

From 'sh ver':
Hardware:   PIX-515E, 32 MB RAM, CPU Pentium II 433 MHz

I don't think you'll get 92 Mb out of m0n0wall on a box of that spec. 
But match them up dollar for dollar (that's a ~$2500 USD firewall) on
hardware and m0n0wall will come out way ahead.  m0n0wall should come
out ahead on less than half the hardware, dollar wise, as a rough

I'm a Cisco nut myself, but I'll take a m0n0wall over a PIX any day.    

> So, I'm asking this list, what testing should we do?  Assume we have
> the facility and ability to do static or dynamic public addresses for
> WAN, NAT or public addresses for LAN, port forwarding, etc.  Pretty
> much any scenario should be workable.  Once we have a good set of
> things to test, and can put together a good test plan, we're both
> interested in doing a good test.

iperf is probably a good tool for the job, but that only tests a
single TCP stream.  That's a good indicator of max throughput, but not
of how scalable the firewall really is (you aren't going to push 92 Mb
through your firewall on a single TCP stream, generally, maybe over a
few thousand TCP connections).  I'd hit Google and see what kind of
other testing tools you can dig up.  I'd definitely be interested in
anything you come up with.