[ previous ] [ next ] [ threads ]
 
 From:  Angus Jordan <angus dot jordan at gmail dot com>
 To:  James Sheridan <stuff at phosphor dash ink dot com>
 Cc:  m0n0wall at lists dot m0n0 dot ch
 Subject:  Re: [m0n0wall] Beta 1.2b10 - All mbuf clusters exhausted
 Date:  Mon, 19 Sep 2005 15:18:04 -0700
Hi James,

On 9/19/05, James Sheridan <stuff at phosphor dash ink dot com> wrote:
> 
> Angus Jordan wrote:
> > Hi James,
> >
> > What are your hardware specs? Also, can you goto the
> 
> iTox mini-ITX motherboard with 64MB memory and the CF card. 4 NICs, two
> 100MB onboard and one dual-GBit card. P3 800(? I forget) CPU.


Sounds good to me, I've run m0n0wall on a smaller machine without any 
issues.

I'm REAL happy with the box for my needs.
> 
> > http://monowall.ip/exec.php and type this command in:
> >
> > netstat -m
> 
> I've included the values from my 1.2b3 setup and the 1.2b10 setup (just
> loaded the image, no tweaks) and included the top output for giggles.
> 
> 
> 1.2b3
> =====
> 
> $ netstat -m
> 676/912/4096 mbufs in use (current/peak/max):
> 676 mbufs allocated to data
> 671/894/1024 mbuf clusters in use (current/peak/max)
> 2016 Kbytes allocated to network (65% of mb_map in use)
> 0 requests for memory denied
> 0 requests for memory delayed
> 0 calls to protocol drain routines
> 
> $ top -d1
> last pid: 271; load averages: 0.00, 0.00, 0.00 up 2+14:40:42 15:44:57
> 16 processes: 1 running, 14 sleeping, 1 zombie
> 
> Mem: 6172K Active, 4176K Inact, 7284K Wired, 4992K Buf, 29M Free
> Swap: 
> 
PID USERNAME PRI NICE SIZE RES STATE TIME WCPU CPU COMMAND
> 111 root 2 0 2512K 1704K select 0:04 0.00% 0.00% snmpd
> 93 root 10 0 1452K 1204K nanslp 0:03 0.00% 0.00% ipmon
> 98 root 2 0 1024K 752K select 0:01 0.00% 0.00% syslogd
> 106 nobody 2 0 960K 736K select 0:00 0.00% 0.00% dnsmasq
> 108 root 2 0 1920K 1588K select 0:00 0.00% 0.00% dhcpd
> 80 root 2 0 1436K 1080K select 0:00 0.00% 0.00% dhclient
> 119 root 2 0 2436K 1436K select 0:00 0.00% 0.00% mpd
> 267 root -6 10 2416K 2124K piperd 0:00 0.00% 0.00% php
> 101 root 2 0 2288K 1616K select 0:00 0.00% 0.00% mini_httpd
> 117 root 10 0 1100K 772K nanslp 0:00 0.00% 0.00% msntp
> 271 root 49 10 1868K 976K RUN 0:00 0.00% 0.00% top
> 115 root 10 0 1328K 792K wait 0:00 0.00% 0.00% sh
> 145 root 3 0 1332K 848K ttyin 0:00 0.00% 0.00% sh
> 270 root 10 10 1328K 820K wait 0:00 0.00% 0.00% sh
> 269 root -6 0 2372K 1836K piperd 0:00 0.00% 0.00% mini_httpd
> 
> 
> 1.2b10
> ======
> 
> $ netstat -m
> 988/1024/4096 mbufs in use (current/peak/max):
> 988 mbufs allocated to data
> 986/1020/1024 mbuf clusters in use (current/peak/max)
> 2296 Kbytes allocated to network (74% of mb_map in use)



Do you have any amount of traffic going through your firewall? How large is 
your environment?

I have an environment with 35 users, no tweaks set on the m0n0wall machine 
with a fairly similar hardware configuration and my mbufs are in the range 
of 20-35% usage....

0 requests for memory denied
> 0 requests for memory delayed
> 0 calls to protocol drain routines
> 
> $ top -d1
> last pid: 200; load averages: 0.03, 0.04, 0.01 up 0+00:03:00 15:50:00
> 16 processes: 1 running, 14 sleeping, 1 zombie
> 
> Mem: 6128K Active, 4416K Inact, 7444K Wired, 24K Cache, 5040K Buf, 29M
> Free Swap:
> 
> PID USERNAME PRI NICE SIZE RES STATE TIME WCPU CPU COMMAND
> 196 root -6 10 3812K 3172K piperd 0:00 2.00% 0.10% php
> 93 root 2 0 1036K 780K select 0:00 0.00% 0.00% syslogd
> 114 root 2 0 2440K 1460K select 0:00 0.00% 0.00% mpd
> 88 root 10 0 1456K 1224K nanslp 0:00 0.00% 0.00% ipmon
> 106 root 2 0 1532K 1232K select 0:00 0.00% 0.00% snmpd
> 110 root 10 0 1332K 840K wait 0:00 0.00% 0.00% sh
> 112 root 10 0 1104K 788K nanslp 0:00 0.00% 0.00% msntp
> 96 root 2 0 2292K 1640K select 0:00 0.00% 0.00% mini_httpd
> 140 root 3 0 1336K 904K ttyin 0:00 0.00% 0.00% sh
> 199 root 10 10 1332K 876K wait 0:00 0.00% 0.00% sh
> 75 root 2 0 1440K 1096K select 0:00 0.00% 0.00% dhclient
> 198 root -6 0 2376K 1872K piperd 0:00 0.00% 0.00% mini_httpd
> 101 nobody 2 0 1012K 780K select 0:00 0.00% 0.00% dnsmasq
> 103 root 2 0 1920K 1592K select 0:00 0.00% 0.00% dhcpd
> 200 root 48 10 1872K 952K RUN 0:00 0.00% 0.00% top
> 
> > Also, for your reading; here is the pertinent info from the tuning 
> manpage:
> >
> > <snip>
> > kern.ipc.nmbclusters may be adjusted to increase the number of network
> > mbufs the system is willing to allocate. Each cluster represents approx-
> > imately 2K of memory, so a value of 1024 represents 2M of kernel memory
> > reserved for network buffers. You can do a simple calculation to figure
> > out how many you need. If you have a web server which maxes out at 1000
> > simultaneous connections, and each connection eats a 16K receive and 16K
> > send buffer, you need approximately 32MB worth of network buffers to 
> deal
> > with it. A good rule of thumb is to multiply by 2, so 32MBx2 = 64MB/2K =
> > 32768. So for this case you would want to set kern.ipc.nmbclusters to
> > 32768. We recommend values between 1024 and 4096 for machines with mod-
> > erates amount of memory, and between 4096 and 32768 for machines with
> > greater amounts of memory. Under no circumstances should you specify an
> > arbitrarily high value for this parameter, it could lead to a boot-time
> > crash. The -m option to netstat(1) may be used to observe network clus-
> > ter use. Older versions of FreeBSD do not have this tunable and require
> > that the kernel config(8) option NMBCLUSTERS be set instead.
> > <snip>
> 
> Sounds like I need to set kern.ipc.nmbclusters to something higher than
> 1024, right? Just idling I'm almost running out and any traffic must
> push me over. With 64MB I'm thinking 2048 should be ok. Thoughts?


2048 would be fine, you may want to play around a little with higher and/or 
lower numbers. Don't just arbitrarly set a high number, as you could suck 
memory from other processes that may need it.

Have a great afternoon.

Thanks,
Angus Jordan