[ previous ] [ next ] [ threads ]
 From:  Didier Lebrun <dl at quartier dash rural dot org>
 To:  <m0n0wall at lists dot m0n0 dot ch>
 Subject:  Re: [m0n0wall] m0n0wall behind a satellite connection
 Date:  Tue, 08 Feb 2005 20:58:02 +0100
At 14:16 07/02/2005 -0600, you wrote:
>Greetings.  I am having some strange issues running m0n0wall on a Soekris
>net4511 behind a direcway satellite modem.  If I connect a computer (with
>DHCP enabled) straight to the modem, the computer gets configured with a
>gateway of a.b.c.d, an IP address of a.b.c.(d+1), and a netmask of
>  From my understanding, this means this end of the
>satellite connection is a subnet with 2 valid IP addresses: the one of the
>modem itself, and the one for your device behind the modem.  Well, when the
>computer is plugged in with DHCP, the system works well.  If I put the
>m0n0wall box behind the modem, web pages either take a lot longer to load or
>do not load at all.  Initially, we had this problem and a call to their tech
>support revealed that they do not recommend setting your end of the network
>to DHCP - they want you to set the addresses statically.  I did this a while
>back and it seemed to work well for a while.  Recently, this problem has
>appeared again.  My first thought was that the ISP changed their DNS
>servers, so I noted the addresses that the computer pulled using DHCP and
>compared them to the addresses set in m0n0wall.  They matched.
>I am mainly wondering if there are some special settings needed for running
>behind a satellite connection due to the extremely high latency.
>thank you


In my small village in France, we have been sharing a bidi sat link through 
a WLAN network for over a year, and we've managed to make it work fine. 
Although our gateway doesn't run precisely under m0n0wall, it's a FreeBSD 
4.8 system quite similar to m0n0wall. Our provider is not DirecWay, but 
Eutelsat, and we have a subnet too between our gateway's 
WAN interface and the sat modem.

The high latency does create some specific problems with TCP, since the 
[bandwidth * latency] product is too big for the standard TCP window size 
and packets loss can become dramatic in this kind of context. But the IETF 
has developped a set of TCP extensions (RFC 1323) in order to overcome 
these limits and have TCP perform better on LFNs (Long Fat Networks). 
Recent versions of FreeBSD support the RFC 1323 extensions and use them 
automatically when necessary, but the main problem is between the clients 
and the remote servers anyway, since the TCP transaction occur between 
ends, the gateway just letting the packets through, with the exception of 
DNS and NTP queries.

- Enhancing TCP Over Satellite Channels using Standard Mechanisms
- TCP Extensions for High Performance
- TCP Research Related to Satellites
- Increasing TCP's Initial Window

Win98 and previous versions do NOT support RFC 1323 natively. If using 
Win98, you must download and install the VTCP patch (vtcp_*.exe) and a NDIS 
patch (ndis_*.exe) (search microsoft.com website). If still using Win95, 
there is no way as far as I know. Later versions of Windows (WinMe or more) 
do support RFC1323 natively, and reasonably recent versions of MacOSX, 
Linux and *BSD too.

When RFC 1323 extensions are supported by the system, TCP adjusts itself by 
calculating the TCP window size as soon as the first ACK comes back... but 
before that, it uses the system default values, which are often badly 
optimized for sat links. So, it's better to tweak them on each client in 
order to have it perform better. The main principles are:
         - enable "Window Scaling" (usually enabled by default)
         - enable "Time Stamping"
         - enable "Selective ACKs" (SACK)
         - enable "Path MTU Discovery"
         - set the TCP receive window size to a higher value [max bandwidth 
in bytes * max latency in sec]
         - disable "Black Hole Detection" (MS Windows only)
         - set "Max duplicate ACKs" = 2 (3 on Win98 only)

On MS Windows, you can use a soft called Dr TCP 
(http://www.dslreports.com/drtcp), which is just a GUI for the registry keys.

On FreeBSD, you can adjust a few thing too:
         Add in /boot/loader.conf
         # -- RFC 1323 tweak -- #

         Add in /etc/sysctl.conf
         # -- RFC 1323 tweak -- #
         #kern.ipc.maxsockbuf=262144 # default is ok
         #net.inet.tcp.rfc1323=1 # default is ok
         net.inet.tcp.recvspace=98304 # set to your download [max bandwidth 
in bytes * max latency in sec]
         net.inet.tcp.sendspace=24576 # set to your upload [max bandwidth 
in bytes * max latency in sec]
         #net.inet.tcp.path_mtu_discovery=1 # default is ok
         net.inet.udp.recvspace=98304 # set to your download [max bandwidth 
in bytes * max latency in sec]

The values indicated above are based on a 128/512 kbits sat link, 
considering a maximum latency of 1.5 ms for 1500 bytes packets flowing one 
way, and 44 bytes packets (ACKs) the other way:
         512 * 1024 / 8 * 1.5 = 98304
         128 * 1024 / 8 * 1.5 = 24576
You must adjust the values to your link.

If you are using FreeBSD's traffic shaping capabilities, you must adjust to 
size of the queues too, in order to avoid packets drops when the queue is 
full. You can set each download queue to the TCP receive windows size, and 
each upload queue to the TCP sendspace. The same for the main pipes 
(96Kbytes and 24Kbytes in our case).

Another kind of problem that can arise, is MTU/MSS miscalculations, since 
the TCP header is 4 bytes longer than usual when using the RFC 1323 
extensions. They fill the 6th optional TCP header line, thus producing 
headers of 44 bytes (20 IP + 24 TCP) instead of 40 usually (20 IP + 20 
TCP). It can create problems when using VPNs or any kind of encapsulation.

Hoping it can help you,

Didier Lebrun
Le bourg - 81140 - Vaour (France)
tél: (AM et soirée)
mailto:dl at vaour dot net (MIME, ISO latin 1)