On Wed, 8 Sep 2004, Michael David Joy wrote:
> Now directly Fedora Core 2 box to Fedora Core 2 box via ftp without
> routing on a D-Link gigabit rack mount switch, we get 55-60 MByte/s.
> Our initial numbers average around 40 MByte/s on multiple runs with
> large multi-gigabyte file transfers of various sizes. The systems do not
> have a problem with reading or writing this fast as they have 2 WDC
> 250GB SATA 7200 RPM drives in a raid 0 config.
> This indicates that there's about a 20 percent wire speed limitation
> somewhere. My bet's on the FreeBSD network layer / drivers for the
> broadcom chip.
How big are your socket buffers? Note that at Gb speeds, each additional
millisecond of RTT needs an additional 125K of socket buffer to avoid
being window-limited. And of course RFC1323 window scaling needs to be
enabled for this to do any good. Even "short fat pipes" need big windows
when they're sufficiently "fat". :-)
Also note that you need at least two packets of buffering per "hop" (i.e
one per round-trip hop), where even going through a switch counts as
another hop if it's operating in the usual store-and-forward mode. This
is probably not an issue with "normal" packet sizes in this setup, but
could be if you're using jumbo frames.
On Thu, 9 Sep 2004, Manuel Kasper wrote:
> On 09.09.2004 11:50 -0400, Mine GO BOOM wrote:
> > That reminds me: why was the load averages removed from the uptime
> > line on index.php? When I was testing the traffic shaper, it helped
> > me out in judging how well it would work under heavy load, since the
> > processor in mine is very weak. I guess I'll just have to go through
> > exec.php now.
> Because the load averages are no good for telling how much CPU time
> was spent in the kernel (where all the I/O and packet filtering
> happens). They're simply misleading on a system like m0n0wall.
More precisely, they don't indicate interrupt time. I believe they *do*
indicate kernel time spent at process level (i.e. system calls), but most
packet processing is done at interrupt level. And any reasonably portable
way of measuring the interrupt time would probably involve more overhead
than one would like to see. Especially if the processor is slow enough
for it to be an issue. :-)