[ previous ] [ next ] [ threads ]
 
 From:  Adam Nellemann <adam at nellemann dot nu>
 To:  Eric Shorkey <eshorkey at commonpointservices dot com>
 Cc:  m0n0wall at lists dot m0n0 dot ch
 Subject:  Re: [m0n0wall] some notes on how we added traffic graphs to m0n0wall...
 Date:  Sat, 03 Apr 2004 03:14:30 +0200
Hi Eric,

I hear you loud and clear. In fact I asked much the same question 
(only, lacking knowledge about *NIX tools, my suggestions for 
alternatives to SVG was different, and probably not as good as yours). 
Feel free to find my (recent) post in the archives and comment on my 
(newbie) suggestions.

In particular I feel that the current way the SVG graph works it is 
next to useless (no offence to whomever made it), since it will 
provide only 20 seconds of data. Aside from being a redicously short 
period, it makes the graph quite "chunky" and the data will often be 
gone before one gets to look at it.

Also, a plugin like SVG is prone to cause trouble for some users, as I 
found out when, for unknown reasons, an older version was installed 
the first time I loaded the traffic grapher page, which caused an 
error in IE. Also there have been some posts about people having 
trouble getting the plugin to work with various (non-IE) browsers.

Btw. Am I correct in assuming that your solution doesn't use a plugin, 
but rather display the graphs as a normal image? In that case there is 
a single (albeit easily rebuffed) argument for SVG: It generates very 
little network traffic, even with a high frame-rate (which is nice for 
those short-period graphs, even if I'd never go as low as 20 seconds). 
Look at my previous posts about this, for a solution to this problem 
when using images.

Personally I run m0n0wall on a PC, albeit with a CF card acting as 
harddisk (to keep the noise and moving parts down), thus I wouldn't 
have a problem with the RAM requirements, since the smallest DIMMs I 
could find among my old hardware was 2x128MB :)

In any case, any kind of traffic grapher (IMHO the current SVG 
implementation as well) should be made a plugin, seing as many people 
won't need or use this feature (actually I think the same could 
possibly be said about the VPN and/or DynDNS features). Making it a 
plugin would also allow a traffic grapher to have such "bloated" 
requirements as you mention, since RAM challenged users could then 
either choose a "leaner" grapher plugin, or omit this feature 
altogether (opting instead for a SNMP solution).

Alternativly, a checkbox on the Advanced page for enabling the 
grapher, could be provided (disabled by default). While this might 
require a reboot, so would a plugin being installed, and anyway one 
wouldn't change this setting very frequently.

Since I guess your method doesn't involve SNMP, it might even be 
possible to extend your graphs to include some per-X (where X could be 
any of the following: host, protocol, firewallrule, shaper-rule, -pipe 
or -queue, to the extent these statistics are readily obtainable of 
course). In fact, I should think most of these could be "simulated", 
as long as it is possible to get the per-firewallrule statistics, 
since you could just make one or more passrules catching the specific 
traffic you'd want to graph.

I really hope that you, or someone else, will consider turning your 
excellent work (perhaps with that RAM disk feature you mention) into 
one or two plugins.

To all persons able to do this plugin: Come on, be the first one to 
make a m0n0wall plugin! I'm sure you will be rewarded with ample 
helpings of respect :)


Regards,

Adam.


Eric Shorkey wrote:

> To be honest, I'm surprised that m0n0wall isn't using rrdtool to generate
> it's graphs. SVG is cool and all, but why go through all that trouble, when
> you could just dump data to rrdtool every few seconds. And since it's a
> round robin database, the filesize is constant, so it won't eat up your
> compact flash card's memory. If you want the graph to update automagically
> every 15 seconds, just use javascript to reload the image at a certain
> interval.
> 
> Of course, there is a downside. RRDTOOL requires the use of an rrdtool
> database, which is just a file that you have rrdtool create for you so you
> have somewhere to store these datapoints. As you collect datapoints, you
> give them to rrdtool to store within that database, and later you use the
> graph generator within rrdtool to create pretty images with lines and colors
> and all that to give you a visual description of whatever data you've been
> having it store all this time. Most compact flash cards are good for 100,000
> writes per block, or something insane like that. Here is where it gets bad.
> That means that as your firewall runs, rrdtool would be constantly writing
> to the same few blocks on your CF card. Which, after a few years, would kill
> it. A better idea would be to have m0n0wall create a ram disk on boot, and
> create the rrdtool database within that. Since you aren't going to be
> storing a lot of datapoints (only 1 day's worth at the most), the ramdisk
> would only need to be a few MB.
> 
> I bet you're wondering why you only store 1 day of graph data at the most.
> If you wanted longer graphs, you should use SNMP to grab the data, and store
> it on another box. Preferably one with a much bigger hard drive. High
> resolution graphs of firewall throughput can quickly turn out to be bigger
> than you had anticipated. For instance: I store our firewall traffic stats
> for 1 year, and poll the device every 15 seconds for data. (This allows me
> to get 1 minute averages over 4 datapoints. If one of the datapoints is
> invalid, corrupt, gets "lost", or whatever, i still have 3 good ones to get
> a proper average from.) These 1 minute averaged datapoints, for only 2
> interfaces, created a 32MB database. I have another firewall with 4
> interfaces in it, and that DB is 64MB.
> 
> So anyways, if you're going to put a traffic graph right into the firewall,
> you're really not going to care much past 1 day's worth of information. You
> really only log into a firewall for 2 reasons. Either you're making a
> change, or your trying to figure out what's broke. You're really only going
> to care about immediate information, like what's been going on over the past
> hour, and the past day. Anything more than that, and you'd be looking at
> your much bigger traffic logs.
> 
> ----- Original Message ----- 
> From: "Adam Nellemann" <adam at nellemann dot nu>
> To: "Jay Wherley (SEI)" <jrw at squires dash eng dot com>
> Cc: <m0n0wall at lists dot m0n0 dot ch>
> Sent: Friday, April 02, 2004 6:32 PM
> Subject: Re: [m0n0wall] some notes on how we added traffic graphs to
> m0n0wall...
> 
> 
> 
>>Hi Jay,
>>
>>Jay Wherley (SEI) wrote:
>>
>>>i've put together some notes and a pair of screenshots describing a
>>>method we used to add network traffic (and net4801 enviromental
>>>data - Temperature, Vcc, Vpower) graphs to m0n0wall.
>>>
>>>full details here: http://www.squires-eng.com/graphinfo.html
>>>
>>>the data is collected (via cron job) using SNMP for the network
>>>interfaces, and some parsing of env4801 program output for the
>>>enviromental data. it is stored in a fixed size "round robin"
>>>database.
>>>
>>>lots of different custom graphs could be used to pull up the data -
>>>the ones i made show hourly, daily, weekly, monthly, and yearly
>>>data.
>>>
>>>the data is persisted across power cycles by periodically copying
>>>the database to /cf (and using that if it exists at startup).
>>
>>VERY nice! Any chance you'd care to implement this using the new
>>plugin interface for m0n0wall, making it a bit easier for us *NIX
>>ignorants to get it up and running?
>>
>>If not, anyone else on the list who'd be up to this task?
>>
>>By the way, I'm using the PC version of m0n0wall, any chance this
>>would work anyway? I guess the environment part wouldn't, at least not
>>without some changes, but I'm primarily after the traffic graphs.
>>
>>Perhaps, if being implemented as a plugin, it could be split into two
>>seperate ones, one for the traffic, and one for the environmental
>>stuff. This would also allow the latter to be modified for the various
>>platforms supported, in case someone want or need to do so.
>>
>>
>>Regards,
>>
>>Adam.
>>
>>---------------------------------------------------------------------
>>To unsubscribe, e-mail: m0n0wall dash unsubscribe at lists dot m0n0 dot ch
>>For additional commands, e-mail: m0n0wall dash help at lists dot m0n0 dot ch
>>
>>
> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: m0n0wall dash unsubscribe at lists dot m0n0 dot ch
> For additional commands, e-mail: m0n0wall dash help at lists dot m0n0 dot ch
>