The m0n0BSD Hackers Guide
(current version: 0.10 / last update to this document: Wed May 14 22:10:21 CEST 2003)

Contributing to m0n0BSD

This guide describes how to boot your soekris from a host system, having the rootpartition remote (NFS) mounted so you can easily edit and add to the monoBSD system.

I know that using NFS is unsecure (to say the least), so if you want to be running and deploying monoBSD in a secure environment, please build a target kernel after you have debugged the root filesystem without NFS. Use this kernel to put on your Compact flash.

Of course you can use the method described in the m0n0wall hackers guide, only using PXE-boot and having the root filesystem in memory. This does not need NFS, but it will need memory.... As m0n0BSD is larger (by nature / design) than m0n0wall, NFS-based development should be the way to go. However: YMMV.

Philosophy

In order to get a development environment for m0n0BSD you will need the target hardware and a host system. typically this will be a soekris NET45xx board and a FreeBSD 4.8-RELEASE (Intel) system.

You will be booting the m0n0BSD from the host sytem, so no CF card is needed. The boot image is located on the host and changes in the kernel or root filesystem are easily made by hacking them on the host machine.

The host kernel

As you will be network (PXE (Preboot Execution Environment)) booting the soekris from your host system and be able to generate image files to load you will have to set up some environment on the host to support just that. First of all, be sure that you have a kernel running that contains the vnode driver. This feature to be able to turn a file into a device. Moreover you will need to run NFS.

Add the following to your kernel configuration and rebuild the kernel:


options         NFS             # Network Filesystem
pseudo-device   vn              # Vnode driver (turns a file into a device)

Instructions for rebuilding the kernel can be found in chapter 9 of the FreeBSD handbook.

The PXEboot environment

The PXE boot environment needs a DHCP server on the subnet and a tftp server on the subnet, reachable by the soekris.

Prepare the DHCP server

Check if you have a (running) DHCP server on the network where the soekris (and the development system) are on. If not, build isc-dhcp from the ports tree with:

cd /usr/ports/net/isc-dhcp3; make all install clean

PXE boot needs some extra entries in the dhcp record for the (soekris) target. It needs the (tftp)server address from which to retrieve the bootfile (denoted by the next-server keyword) and the actual name of the bootfile (denoted by filename). Also it will need the path to the root filesystem An example of the dhcp record for the soekris is shown below. Here 192.168.100.100 is the ip number of the soekris and 192.168.100.2 is the ip number of the host that provides dhcp and nfs.:

host soekris {                
     hardware ethernet 0:0:24:ad:bc:ac;                                     
# ip of soekris box
     fixed-address 192.168.100.100;
     filename "pxeboot"; 
# ip of (nfs) server
     next-server 192.168.100.2;
     option root-path "192.168.100.2:/mboot";
 }

Prepare the tftp server

To enable the tftp server to be started from inetd (you should be running inetd, did I mention this ??) uncomment the following line in /etc/inetd.conf:

tftp  dgram  udp wait root /usr/libexec/tftpd  tftpd -s  /tftpboot
Now, restart inetd:
kill -HUP `cat /var/run/inetd.pid`

Prepare the nfs server

To enable the nfs server you should be running the portmapper.

In /etc/rc.conf change the following lines:
nfs_client_enable="YES"
nfs_reserved_port_only="YES"
nfs_server_enable="YES"
Create an exports file in /etc/exports, (here I assume 192.168.100.100 as the ip address of the target (soekris) as defined in dhcp) and restart mountd:
cat >> /etc/exports
/mboot -ro -maproot=0 192.168.100.100

kill -HUP `cat /var/run/mountd.pid`
Now, if nfs was not running reboot to cleanly start up the nfs daemons.

Setting up m0n0wall on the host

Extracting the rootpartition

The root file system for m0n0BSD can be found in the image that you can download from: http://www.m0n0.ch/bsd/ To extract it, and write it to a directory that you can NFS export to the soekris. Mount the image under /mnt using the vnode driver, and copy the filesystem over to e.g. /mboot. Proceed as follows:
mkdir /mboot
vnconfig -s labels -c vn0 ./m0n0bsd-4.8-net45xx-std-xxxxxxxx
mount /dev/vn0a /mnt
tar cf - /mnt | tar xpvf - -C /mboot 
cd /
umount /mnt
vnconfig -u vn0

Now /mboot on the host contains the rootdirectory of your target system and can be nfs-mounted. Because we are using nfs, we will need one extra binary in /sbin. This binary is not part of the default monoBSD image. Mount_nfs is used to (re) mount the root partition in order to write to it from the soekris. You can just copy this binary over from the host /sbin:

cp /sbin/mount_nfs /mboot/sbin

The Kernel

In order to get m0n0wall to nfs boot we have to recompile the kernel for m0n0wall using the kernel config file as found on http://www.m0n0.ch/bsd/downloads/M0N0BSD_NET45XX_IPSEC. Make some changes to the kernel config file to add bootp and nfs. Also if applicable change the default behaviour of ipfilter to NOT block. (if you are using ipfilter, the default kernel does not include ipfilter). You need the network to mount the rootpartition during bootup, so if your filter is blocking all traffic it will not work.

ADD:

  
options         BOOTP
options         BOOTP_NFSROOT
options         BOOTP_COMPAT
options         NFS                     #Network Filesystem Client
options         NFS_ROOT                #NFS usable as root device

REPLACE

  
options               IPFILTER_DEFAULT_BLOCK
WITH
  
options               IPFILTER_DEFAULT_ACCEPT
Place this config file (M0N0BSD_NET45XX_IPSEC) in /usr/src/sys/i386/conf. Now build the kernel:
  
cd /usr/src/sys/i386/conf; config M0N0BSD_NET45XX_IPSEC
cd /usr/src/sys/compile/M0N0BSD_NET45XX_IPSEC; make depend all          
strip kernel         
strip --remove-section=.note --remove-section=.comment kernel
copy the kernel file to /tftpboot
cp kernel.gz  /tftpboot

The Modules

cd /usr/src/sys/compile/M0N0BSD_NET45XX_IPSEC; make modules
Then, move the needed modules to the modules directory in the m0n0wall root filesystem. Be sure to use only the modules you need for your application, otherwise you will be ending up with an enormous footprint.

These newly-built modules can be found in /usr/src/sys/compile/M0N0BSD_NET45XX_IPSEC/modules. modules directory)

The /tftpboot directory

The /tftpbootboot directory needs the pexboot file and the kernel.

cp /usr/src/sys/boot/i386/pxeldr/pxeboot /tftpboot

GO

Now boot the stuff....

Remember to turn on dhcp (if needed):

/usr/local/sbin/dhcpd

Also, be sure that you allow nfs from your host system (firewall rules...). You can test this by trying to mount the exported rootpartition on another system (you should add its ip# in the exports file..)

.Now you can test you m0n0BSD system. You will be getting an error in mounting the flash that you just do not have, (CAN'T CHECK FILE SYSTEM) just type control-D at the shell prompt and booting will continue.

Configuring m0n0BSD...

create the rootpassword entry (to prevent the chicken-and-egg-problem (needing the rootpassword to change the rootpassword)).. copy the encrypted rootpassword string found in /etc/master.passwd to the /mboot/etc/master.passwd, and re create the databases:

pwd_mkdb -d /mboot/etc /mboot/etc/master.passwd

Now, reboot the soekris, you will be getting errors when trying to set up the hostname etc (running rc.initial). To overcome this, allow rw mounting of the rootpartition by editting the /etc/exports file, removing the -ro and restart mountd.

Now log in at the soekris as root, using the same passwd as the host, and mount the rootpartition read-write:

mount -uw <serverhostip>:/mboot /
Now, start /etc/rc.initial and configure.

Remember to reset the rootpassword and set a fixed ip address for the soekris. The ip address needs to be the same as the address you set with dhcp. This prevents losing the nfs (rootpartition) when starting.

Some last notes...

If booting does not work, take out your favorite network sniffer (eg. ethereal or tcpdump) and have a "look" on the wire. You should see the dhcp request from the soekris and the offer from the host, then you should see pxeboot loading and the tftp request for the kernel, again a DHCP request, NFS mounting the rootpartition etc.

Be sure to test, test test and test before flashing and deployment ....

A console transcript of an NFS boot of m0n0BSD on a Soekris NET4501 with an additional wireless PCI card can be found here. Note that the first time boot looks different as rc.initial starts.

Building a Compact Flash card / image

After you have created your latest and greatest version of m0n0BSD, you might, after you have documented and published your additions and changes, make it flashable to a CF card. Here's a short description how to proceed:

You will be putting the rootpartition you've just created, together with a (compressed) kernel file onto the flash. Now re build your kernel without NFS support and put the compressed kernel file in the root of the NFS-filesystem (/mboot/kernel.gz). This is what we will flash. The bootloader in the distribution is set up in such a way that it will load the kernel.gz file upon boot.

The procedure for preparing the CF card in the soekris itself or create an off-line flashable image is described in Manuels miniBSD page (Chapters 10 .. 13). Here some quick highlights:

First rebuild the kernel without NFS support, so remove the NFS related lines you have added, and rebuild. Refer to the above section on building the kernel. Strip and compress (gz) the image and copy it in the rootpartition (/mboot instead of /tftpboot as described earlier for NFS) als kernel.gz.

Then, see what the approximate size of the rootpartition is:
du -kc /mboot | grep total
This will give you the size of the root in 1K blocks, make a note of this number. You will need a rootpartition on your flash card with at least this net size. Now take out your CF card, make sure that it is bigger that the size you need. Pop the card in the soekris, and boot from the server once more. (use the boot F0 command in the monitor, or it will try to boot from your (fresh) flash). To determine the layout of your flash, use
disklabel -rwn ad0 auto | grep sectors/unit
If this does not work, erase all partition information from the flash, and try again:
dd if=/dev/zero of=/dev/rad0 bs=1k count=20
disklabel -rwn ad0 auto | grep sectors/unit
OK, now you should know the size of your card and the minimum size of your rootfs. We will be building an image file on the host on the that meets both requirements, and copy in the /mboot rootpartition: (make sure you have got enoungh diskspace on the partition you put the inagefile on...)
dd if=/dev/zero of=/tmp/my-m0n0BSD.bin bs=512 count=<#of sectors on flash>
vnconfig -s labels -c vn0 /tmp/my-m0n0BSD.bin
disklabel -Bwr vn0 auto ; disklabel -e vn0
Now you will be dropped in your favorite (?) editor. Find the line that starts with c:, duplicate it, change c: to a: and set the FSTYPE to 4.2BSD. Make sure the size of this partition is at least the size of your root fs + 10% and less or equal to the total size of the CF card. You can add extra partitions as well. Save, and newfs, mount, copy the rootfs in and umount again.
newfs -b 8192 -f 1024 -U /dev/vn0a
mount /dev/vn0a /mnt
tar cf - /mboot | tar xpvf - -C /mnt 
cd /; umount /mnt
Now your file (in /tmp/my-m0n0BSD.bin) is ready to be flashed. An easy road is, copy the flashfile in the (NFS) rootpartition on the host serving your still running soekris (you did not turn it off after checking the flash size did you ?), and dd the flashfile onto the CF card using the soekris. Ofcourse you can use your favorite tools to do it any other way.
dd if=/my-m0n0BSD.bin of=/dev/rad0 bs=8k

This will take some time, depending on the size of the file and the flashcard brand (write-speed). using <ctrl> T will give you some progress info. Do not forget to remove the image file from your remote mounted root if finished...

Now, if all is done, RESTART THE SOEKRIS and watch it boot from your just created flash, and enjoy the new features you just added !

Your soekris, running your own customized m0n0BSD is ready to be deployed !

Having your own compact flash based soekris running now, you can ofcourse make changes to the rootpartition by remounting it RW, change and remounting it RO again. The disadvantage here is that you will not have those changes in your image for deployment in other soekri. This method may be good for configuration and test, but not for adding structural features in the distro, you'd be better to do this while network booted and create a new flash image. Again here YMMV.

Thanks...

This m0n0BSD hackers guide is a brief introduction into "roll your own m0n0BSD" just to give you a hand how to include features and rebiuld the image . Any additions and hacks are greatly appriciated. Please report them back, so the "official" m0n0BSD distribution will be growing better and better as the community grows.

As this guide is a transcript of my efforts to get a new m0n0BSD image, it will not be complete, neither will it be absolutely correct (I think), and there will be a different (and problably better) way to do it, so if you have any additions, or find errors / omissions etc, please report them !

Ofcourse all this info is WITHOUT ANY WARRANTY. So: if you mess up your system, flashcard or something else, we are not liable... please be warned !

Thanks:  
  Manuel Kasper          (http://www.neon1.net/)        
  Dirk-Willem van Gulik (http://www.webweaving.org/wlg/)

© 2003 by Rudi van Drunen <R.van.Drunen@xs4all.nl>. All rights reserved.