[ previous ] [ next ] [ threads ]
 From:  Fred Wright <fw at well dot com>
 To:  m0n0wall dash dev at lists dot m0n0 dot ch
 Subject:  Re: [m0n0wall-dev] FreeBSD GigE Tuning
 Date:  Sat, 11 Sep 2004 01:06:50 -0700 (PDT)
On Fri, 10 Sep 2004, Michael David Joy wrote:
> > How big are your socket buffers?  Note that at Gb speeds, each additional
> > millisecond of RTT needs an additional 125K of socket buffer to avoid
> > being window-limited.  And of course RFC1323 window scaling needs to be
> > enabled for this to do any good.  Even "short fat pipes" need big windows
> > when they're sufficiently "fat". :-)
> I'm not sure what my socket buffers are. The box is in a plain vanilla
> m0n0 pc config. Whatever configuration the default cdrom image comes as
> is what it's configured as. 

I was referring to the socket buffers at the endpoints.  Socket buffers in
intervening routers are irrelevant.

> For the record, I'm not using jumbo frames (most of our network here is
> jumbo fragmenting in nature), and I'm not sure whether the RFC1323 is on

Of course without jumbo frames, you easily get eaten alive by the
per-packet processing overhead.  To keep up with Gb Ethernet with
1500-byte packets, you need to do all the processing and forwarding of the
packet in 12.3 microseconds.

On Fri, 10 Sep 2004, Michael David Joy wrote:

> Well, after some in depth research, I came across this information for
> tuning the FreeBSD net stack.
> There's a tuning patch we might want to add, or at least investigate for
> the 5.1 release.
> http://www-didc.lbl.gov/%7Ejin/network/lion/patches/smbuf.patch.tgz

Again, this is an endpoint issue, not a router issue.  As is the rest of
the stuff in this area.

On Fri, 10 Sep 2004, Michael David Joy wrote:

> After increasing the tcp receive window and buffers to 1+ MB on the m0n0
> box, and 3 and 8mb on the linux boxes, there is no speed increase.
> This leads me to believe that the problem is elsewhere in the kernel.

Did you increase the send window as well (at the sender)?

Can you capture packets at this speed?  With a separate sniffer, perhaps?

But bear in mind that "serious" routers usually use OSes that are designed
to be routers, unlike Unices that are designed to be hosts that happen to
be able to do routing.

					Fred Wright