[ previous ] [ next ] [ threads ]
 From:  Marc <telenieko at gmail dot com>
 To:  m0n0wall dash dev at lists dot m0n0 dot ch
 Subject:  Re: [m0n0wall-dev] Re: [m0n0wall] The future
 Date:  Fri, 28 Oct 2005 11:04:38 +0200
Oh! there was one think we didn't care of....

Currently if you want to touch anything in m0n0 you need a FreeBSD due
to the MFS approach, most development of m0n0 once you have the
contents of mfsroot is platform independent as you can boot it from
pxe for testing.

In my case for example I really hate changing anything of m0n0 'cause
I have to boot a FreeBSD on a VM, the mount the mfs on it change it,
umount, send it to my linux box and then do whatever I want with it.

Are there any real possible alternatives to the MFS approach? or at
least the UFS filesystem, that way the development of m0n0 would be
much more platform independent what could attract more developers!


On 10/28/05, Jim Thompson <jim at netgate dot com> wrote:
> I wasn't going to respond until Chris did. Some of what Fred says is
> just plain wrong.  Unlike Chris, I'm not afraid of having my head handed
> to me, thought I doubt it will happen in this forum.
> Please forgive what seems like a "lashout" below.  Its not intended as
> same, but some of Fred's statements of "truth" are really unfounded
> opinions.
> Fred Wright wrote:
> > Here are my thoughts:
> >
> >System Specs:
> >-------------
> >
> >Embedded platforms (e.g., Soekris & WRAP) should remain the primary target,
> >regardless of what surveys say.  Even if lots of people want to run
> >m0n0wall on boxes where the CPU lifetime after a fan failure is measured in
> >milliseconds, those who are astute enough to use more reliable hardware
> >without moving parts shouldn't be neglected, any more than Mac, Linux, and
> >BSD desktop users should be neglected just because 95% of the desktops run
> >Windows.
> >
> >
> I'd advocate adding those VIA CPU platforms which run fanless for
> extended periods.  Even the dual 1GHz CPU "VIA DP-310" board (with 1 x
> 10/100/1000 and 2 x 10/100 Ethernet) can run fanless.  Neither is VIA
> the only manufacturer of boards with their chips on them.   Picking up
> support for using Padlock to accelerate IPSEC would be a very good thing.
> Should m0n0wall ever move to netbsd or openbsd (I'm not avocating
> either), several other CPU architectures could support the 'fanless'
> requirement.
> >The "fanless" requirement imposes a serious upper limit on CPU power which
> >is *not* significantly alleviated by "Moore's law" evolution (especially
> >with the x86 architecture).  Thus it's not reasonable to allow slothful
> >code and assume that the performance problems can be covered up by throwing
> >faster hardware at it.  And of course, something which runs well on slower
> >hardware will run really well on faster hardware, while something that runs
> >adequately on faster hardware may be unusable on slower hardware.
> >
> >
> In general this is true, but it also possible that something which limps
> on 486 class processors (such as the Elan)
> might *fly* on PII and up.   DES and AES are fine examples.   For
> example, Rijndael (the algorithm in AES) runs at 330KBps on a 50MHz
> 486DX2, but 4.29MBps on a P90, and 12.88MBps on a 200MHz PII.   (128
> bit  keys/128 bit messages.)
> You can consider an Elan 520 to be (effectivly) a 486 DX4 @ 100-133Mhz.
> More telling is the "clock cycles per block"
> 486DX2 @ 50MHz  2370 cycles/block
> P90: 320 cycles/block
> PII @ 200MHz:  237 cycles/block
> >Today, CF is available (at reasonable cost) in sizes pretty much as big as
> >one would conceivably need for this sort of application, while RAM on the
> >embedded platforms remains fixed and limited.
> >
> I'd take it one step farther.   Try (just try) buying CF cards smaller
> than 128MB in any volume (we buy 200-500 at a time).
> Thus, I think the "m0n0 is 8MB (or less)" limit needs to be relieved some.
> >After a successful update, the roles get switched.  There are already some other devices that use
this approach.
> >
> >
> I have code to do this (based on nanoBSD) if someone wants it.
> >Although "object-oriented" seems to be many people's favorite bandwagon at
> >the moment, the baggage that OO languages bring along isn't really
> >appropriate for something like a firewall.
> >
> For the languages that most people use (C++, Perl, etc) OO is just
> syntactic sugar.
> >In a security-conscious application, you really don't want a bunch of complex, obtuse, and often
buggy mechanisms hiding behind the language definition.  And when one uses
> >an OO language, there's a tendency to frame everything in OO terms, whether
> >there's a real benefit or not, due to the "when you're a hammer, everything
> >look like a nail" principle.  Programmers who can't blow their noses
> >without a box of "object-oriented" tissues shouldn't be doing firewall
> >programming.
> >
> >
> Sticking with low-horsepower boxes and avocating OO is  a lot like
> shooting yourself in the foot.  Anyone looking at the runtime
> requirements for C++ or Perl with an embedded system as a target should
> be continually fighting the gag reflex.
> >PHP has serious performance issues due to being an interpreted language,
> >but that can be tolerated in the WebGUI, particularly if more attention
> >were paid to performance issues.  I'd certainly get away from using it for
> >non-GUI purposes.
> >
> >
> Its OK as a 'runs-at-boot' configuration interpreter, but yeah, for
> anything that is CPU sensitive, its the wrong solution.
> However, we do want to minimize the number of language runtimes availabe
> on the platform.
> >1) Polling.  The purpose of polling is to avoid wasting CPU time in
> >interrupt servicing, in order to make more CPU available to userland code.
> >It does *not* have a significant effect on routing throughput (unless the
> >NIC driver is pretty brain-damaged), since the number of packets handled
> >per interrupt tends to self-regulate to the point of "keeping up".
> >
> The opposite has been found in projects such as Click and Xorp.  (I can
> point to other research as well.)
> http://pdos.csail.mit.edu/papers/click:tocs00/paper.pdf
> http://info.iet.unipi.it/~luigi/bsdcon01/polling/mgp00001.html
> On FreeBSD 4.11 Polling increased throughput 4X.  (On Linux polling
> increased throughput by an order of magnitude.)
> >Because of this, it's not possible to determine maximum throughput merely by
> >extrapolating CPU usage from lighter loads.  Thus, on a router, polling
> >only helps ancillary functions (e.g.  the WebGUI) and userland-routed
> >protocols like OpenVPN.  Meanwhile, polling-induced latency is much more
> >significant in a router than in a host, for a number of reasons.
> >
> >
> Please see my comment above, and review the XORP/Click papers (some of
> which really avocate FreeBSD, btw).
> >2) Tuning.  Recently someone posted an excerpt from the FreeBSD tuning
> >guide related to mbuf sizing.  But the stuff about mbufs needed per
> >connection was all based on sizing *socket buffers*, not forwarded
> >packets.  In fact, in a router, once you have enough buffering to avoid
> >starvation, additional buffering actually *worsens* performance by
> >increasing latency (due to longer queue lengths).
> >
> >
> Only true if you allow queue lengths to grow unbouned.  There are times
> during a flow where the extra buffering will help, rather than harm
> throughput.
> >3) Network performance.  Practically all references to "network
> >performance" (including benchmarks) are oriented to *server* performance,
> >not *router* performance.  And claims like "fastest TCP/IP stack" are
> >pretty meaningless when most of the "stack" isn't even involved in routing.
> >
> >
> >On OS Choice:
> >-------------
> >
> >First of all, it should be noted that *no* *nix system is really ideal for
> >routing purposes, for two main reasons:
> >
> >1) The networking code in *nix systems is designed primarily for the host
> >role, with routing added as an afterthought.  The code is designed
> >primarily for originating and receiving packets.  A router OS should really
> >make packet *forwarding* the fundamental activity, with the local system
> >simply being one of the "interfaces".
> >
> >
> Which is why projects such as XORP and Click exist.
> >2) Because *nix context switching is so "heavyweight", it's necessary to do
> >a lot of things at interrupt level just to get decent performance.  In a
> >well-designed microkernel system, context switching is fast enough to do
> >practically everything in userland with good performance.  This improves
> >robustness, debuggability, and CPU allocation.
> >
> >
> Please provide examples that support this assertion.
> >       FreeBSD:        Servers
> >       NetBSD:         Portability
> >       OpenBSD:        Security and Reliability
> >       Dragonfly       Desktops
> >
> >Note that *none* of these actually set out to be optimized as a router OS.
> >However, OpenBSD's emphasis on security and reliability made it an
> >attractive choice for firewall/router applications, which led to greater
> >developer attention to router issues, which led to increased use as a
> >router, etc.  etc.  Thus, it's become the de facto BSD of choice for router
> >applications, which receive a lot of developer attention, including the
> >creation of things like pf and carp.  In general, most *nix-based routers
> >use either Linux or OpenBSD, with the former choice being more due to the
> >"bandwagon factor" than real benefit.
> >
> >
> Juniper runs FreeBSD.  True this is the control plane, not the
> forwarding plane, but where are OpenBSD's "security" bennefits?
> XORP primarily targets FreeBSD (4.11)
> OpenBSD's team is far too willing to violate RFCs in their persuit to
> "rule the world" in my (not so) humble opinion.  Have you looked at the
> 'ntp' implementation they ship?
> Have you *looked* at the mess they made of 802.11 support in OpenBSD?
> >The effect of FreeBSD's "focus" is quite apparent in the 5.x fiasco.
> >FreeBSD's main goal is to provide the best-performing server OS on x86
> >hardware, and it's done an excellent job of that (though recent Linux
> >versions are giving it a run for its money).  But in some cases,
> >server-oriented improvements inflict collateral damage on the routing code,
> >as in 5.x.
> >
> Above you say they're not connected, now you say they are.
> >There's no reason to suppose that this was a one-time thing -
> >the FreeBSD developers are always going to pay more attention to server
> >issues and router issues, because *not* doing so would distract them
> >from server improvements.
> >
> Nor can we say that OpenBSD will continue to be able to claim itself as
> the "most secure".  Past performance is no indicator of future trends,
> and all that.
> >NetBSD would provide the largest choice of hardware platforms, but it's not
> >clear that there are any platforms significantly interesting as routers
> >that are supported by NetBSD and not OpenBSD.  Sure, there may be a few
> >people wanting to run m0n0wall on their dual-NIC toasters, but that doesn't
> >seem like a very good justification.  NetBSD is claimed to offer
> >performance somewhere between FreeBSD and OpenBSD, but those are
> >host-related claims.  And at least in V1.5.3, I found NetBSD to have the
> >worst installer and the most "pickiness" about the hardware configuration
> >of the three (never bothered trying Dragonfly).
> >
> >
> But the 'installer' is a *host issue*, no?
> >Thus, I consider OpenBSD to be the best choice, not primarily due to
> >"better security" (although that's a factor), but because it's the one
> >branch of BSD where the developers take routing applications seriously.
> >
> >The two main arguments that have been advanced against OpenBSD are:
> >
> >1) Performance.  Some have made, and others have parroted, claims that
> >OpenBSD has substantially worse "networking performance" than FreeBSD.  But
> >AFAIK these are all based on *server* performance.  The only *routing*
> >performance comparison I've seen mentioned here was what Manuel did before
> >starting m0n0wall, which was of course several versions ago for both
> >systems.  I've not seen any information on routing performance comparisons
> >between *current* versions of OpenBSD and FreeBSD.  Plus, due to the "focus
> >factor", one would generally expect that comparison to shift more toward
> >OpenBSD as the two systems evolve.
> >
> >
> Just as OpenBSD's current advantages will tend to converge into FreeBSD.
> >2) Atheros support.  While the concept of a vendor-supplied HAL makes
> >sense, Atheros really screwed it up by providing it in binary-only form
> >(for a pretty lame reason AIUI).  If they had done it right - i.e.
> >providing the HAL as CPU- and OS-independent C source, then there would
> >have been no issue.
> >
> I'm certain (as in, I've asked and gotten answers) that they would have
> prefered this path as well.  It wasn't an option, due to the very simple
> fact that it is *trivial* to tune the Atheros chipset *way* out of the
> ISM (and U-NII) bands.   This, in effect means that an "open HAL" would
> (not could, *would*) result in substantial interference issues with
> users of licensed spectrum.   While I could detail the various license
> holders who could scream (and sue) over this, I will point out one very
> vocal set of "licensed" users who would pull every trick in the book the
> second an "open HAL" resulted in detectible interference:   HAM Radio
> Operators (especially the ARRL).  You see, the (nearly world-wide) 13cm
> band runs from 2304MHz to 2450Mhz.
> > As it is, some people grudgingly accepted the
> >"official" HAL, while someone else developed an open-source replacement HAL
> >by reverse engineering Atheros's, and that was introduced in the latest
> >version of OpenBSD.
> >
> And it... doesn't work.  Its total crap.  Have you tried it? Moreover
> its clear (for those of us who can look at both the openbsd and "closed"
> HAL source code that there is clear copyright infringement in the
> (supposed) "reverse engineered" HAL in OpenBSD.
> Want to get (your customer's) sued?  Run the "reverse engineered" HAL.
> >You'll find things of the form "It's evil because it's
> >based on reverse engineering" and "Is this really legal", as well as "When
> >is OpenBSD's open-source HAL going to be ported to our platform so we can
> >junk the binary-only crock?".  I have yet to see any actual behavioral
> >comparisons between the two.
> >
> I have.  I've run them.  We *actively refuse* to support people running
> the "open HAL" code on products we sell.
> > Of course the best scenario would be if the open-source HAL became sufficiently popular to shame
Atheros into providing the "official" HAL in source form.  :-)
> >
> >
> Its not going to happen.   First it would have to work, then the authors
> of the "open source HAL" would end up in court, having to defend
> themselves from a  copyright infringement suit.  Then they would end up
> in court having to defend themselves from the wrath of "spectrum
> licensees" and the regulatory agencies that granted (or sold) those
> licensees.
> >Note that if there really were a significant advantage in using the binary
> >HAL in OpenBSD, it could be used with a "wrapper" in the same manner as it
> >is in FreeBSD.
> >
> Too bad OpenBSD screwed with net80211, or one could merely "replace" one
> with the other.
> >One could even do a side-by-side comparison of two builds
> >of OpenBSD, one with the binary HAL and one with the open-source HAL.  But
> >I wouldn't go to that much trouble without determining that the effort is
> >warranted.
> >
> >
> I've run OpenBSD and FreeBSD on two CF cards on the same hardware
> (soekris and wrap boards) with the same radio cards.   OpenBSD has a
> hard time even maintaing associations.
> >More generally, it's important to make a distinction between what devices
> >are "officially" supported by OpenBSD, and what devices are supported by
> >third-party drivers.  OpenBSD is more rigorous about the definition of
> >"open-source" than some other systems, and hence can't officially include
> >drivers that aren't fully open-source.  I'd expect the same to be true of
> >Linux due to GPL requirements.
> >
> Non-GPL drivers will get more difficult to incorporate into linux over
> time.   this is especially true as the strings are
> tightened on object-only drivers that have to semi-obviously make
> reference (and incorporate) GPL liccensed header files, data strcutures,
> etc.
> A carefull analysis of the Atheros-supplied HAL will show an elegant
> method for avoiding that mess.
> But I digress.
> > While I don't consider an open-source
> >system to be "tainted" by the inclusion of binary *firmware* to be loaded
> >into peripherals, if any code executed by the CPU isn't open-source, then
> >the system can't be honestly described as open-source without a suitable
> >disclaimer.  And one has to question the wisdom of including "black-box"
> >code in a firewall.
> >
> >It's also worth noting that binary driver code is at the very least
> >CPU-specific, making it a major hassle for an OS that supports multiple CPU
> >architectures.  FreeBSD tolerates this more easily due to being, for all
> >practical purposes, x86-only, but other OSes have more difficulty with it.
> >And there seems to be quite a bit of interest in running m0n0wall on
> >non-x86 router hardware.
> >
> >
> The Atheros HAL object is identical (even in binary form) between linux
> on x86, netbsd on x86 and freebsd on x86.  There is an OS-dependent
> 'wrapper' for it.
> For netbsd, mach (as used in Apple's product) and linux the same is true
> for PowerPC (albeit with big/little endian being different
> architectures. 64/32 bit would be an issue if NetBSD and linux didn't
> both run the kernel in 32-bit mode.  (Mach too.))
> Reducing to NetBSD and Linux, this is still true for
> ARM9 (big and little endian)
> Xscale
> MIPS big and little endian
> Your point (as far as it relates to the "closed source" (Atheros
> supplied) HAL is moot.
> >OpenBSD and NetBSD are sufficiently similar internally that drivers (and
> >even in some cases kernel patches) can usually be ported from one to the
> >other fairly easily.  Thus, any device supported by NetBSD can be at least
> >unofficially supported by OpenBSD without too much difficulty.  FreeBSD
> >is sort of the "odd man out" in this department.
> >
> >
> This is mostly because Theo "forked" OpenBSD after the NetBSD guys
> kicked him out for being ... an asshole.
> Only *then* did we start to get "security" as a focus.
> >The bottom line is that the two major arguments against OpenBSD are at the
> >very least lacking in data, if not simply wrong.
> >
> >
> Try again.  Please.
> >Issues with OpenBSD:
> >--------------------
> >
> >Here are some issues I'm aware of that would need to be addressed in moving
> >to OpenBSD:
> >
> >1) FTP NAT performance:  Unlike IPFilter's in-kernel FTP proxy, pf relies
> >on the userland ftp-proxy program.  As far as the control connection goes,
> >this is a big win, since handling the control connection at the packet
> >level is a horribly kludgy and fragile mess.  So much so that Cisco has
> >fixed bugs in that area within the past year.  And control-connection
> >performance is a non-issue.  However, ftp-proxy also proxies the data
> >connection through userland, which involves a significant performance
> >penalty.  It can avoid that in two out of the four combinations of
> >client/server and active/passive, but not the remaining two.  There's an
> >(unofficial) alternative called "ftpsesame", which manipulates the filter
> >tables to forward the data connection properly, but it has a horrible
> >packet-oriented bpf-based kludge for the control connection.  Ideally, one
> >would like a hybrid of the two approaches, but AFAIK none such exists ATM.
> >Note that this is not strictly an OpenBSD/FreeBSD issue, but rather a
> >pf/IPFilter issue, so pfsense is presumably already living with the extra
> >overhead, and since practically everyone seems to agree that it makes sense
> >to move to pf instead of IPFilter, it's an issue regardless of OS choice.
> >Fixing it might be a bit easier in OpenBSD, simply because it's the "native
> >habitat" of both pf and ftp-proxy, without "compatibility layers" getting
> >in the way.
> >
> >
> This is just arrogant.  pf is fully supported in FreeBSD 6 (and above?).
> >Advantages of OpenBSD:
> >----------------------
> >
> >1) Strong orientation toward router-related issues.
> >
> >2) Proactive security.  Some have described this as a "solved problem",
> >but it doesn't remain "solved" as long as the code keeps changing.
> >
> >
> But as this code chances, the other two "BSDs" tend to pick up the
> changes, "quickly".
> Not all security issues are addressed first in OpenBSD.
> >3) Most aggressive development in cryptographic areas, including support
> >for hardware accelerators.
> >
> >
> Nevermind that Sam Leffler is the primary author of many of these, and
> he develops them for FreeBSD first.
> >4) Widely desired features like pf and CARP in their "native habitat".
> >
> >
> Now a non-issue.
> >5) Ability to run on non-x86 hardware.
> >
> >6) IPv6 support that's actually usable.  At least as recently as 5.3,
> >FreeBSD had conflicts between IPv6 and FAST_IPSEC which could cause kernel
> >panics.  Although IPv6 isn't exactly taking the world by storm, it
> >definitely needs to be in any reasonable plan for the future.
> >
> >
> Could we discuss 6.0 or better, please.  Especially given that neither
> m0n0wall or pfSense are likely to be based on any 5.x release of FreeBSD?
> >7) Improved IPsec support, including NAT-T.
> >
> >8) Development cycle based on conservative goals, where new releases have
> >a high probability of being adoptable without significant problems.
> >
> >
> >
> Anyone who has done software development for as long as either of us
> knows that this kind of statement can't stand for long.
> Jim
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: m0n0wall dash dev dash unsubscribe at lists dot m0n0 dot ch
> For additional commands, e-mail: m0n0wall dash dev dash help at lists dot m0n0 dot ch