On Mon, 28 Mar 2005 18:24:49 -0500, George Bourozikas
<george at bourozikas dot net> wrote:
> Out of curiosity, are people using IPsec with m0n0 regularly and without
> problems?  

Tons.


> Am I the only one who finds it impossible to configure IPsec
> properly?
> 

MTU/large packet issues seem to be a frequent problem, though I've
never experienced them and I use a bunch of IPsec tunnels on a number
of different ISP's and different connection types, all with m0n0wall,
some with various other end points (Cisco, Sonicwall, and a couple
other devices).

We'll assume path MTU discovery (PTMUD) is enabled on the host (I
believe it is by default on pretty much every OS).  If it isn't, the
OS should fall back to the default MTU of 576 bytes (in which case you
wouldn't be having this problem).  With PMTUD enabled, the DF (don't
fragment) bit will be set on every outgoing packet.

What should happen when you send a packet that's too big to cross the
VPN without fragmentation is m0n0wall will send back an ICMP
unreachable, "fragmentation needed and DF bit set" message, which
should contain the next hop MTU per RFC 1191.

Anyway, I'm guessing racoon or whatever would handle the ICMP
unreachables on the IPsec connection isn't doing it properly (not
taking into account the IPsec overhead possibly).  I messed around a
little bit and couldn't confirm or deny that.  Since you can easily
replicate the problem, put a sniffer on both sides of the connection,
find the offending packet (or generate some to find the biggest
that'll go through) and see whether an ICMP frag needed and DF bit set
message comes back.  Could be that PMTUD on m0n0wall isn't taking
IPsec overhead into account properly, or it could be that a firewall
on one of the host systems is dropping these ICMP unreachables.

Or, IPFilter has some sort of MSS clamping, though I really don't know
what it does or how it works (other than assumptions from the
denotation).  This may also have something to do with it.

Hopefully Fred Wright or somebody will step in and smack me down if
I've misspoke.  :)  Or fill in some details.  This is getting pretty
deep in the internals of how these things work.

-Chris