[ previous ] [ next ] [ threads ]
 From:  Jim Gifford <jim at giffords dot net>
 To:  m0n0wall at lists dot m0n0 dot ch
 Subject:  need help with DMZ setup, and broken ipsec tunnels
 Date:  Wed, 3 Mar 2004 14:16:52 -0500
A little background:

 I've been playing with lots of the features of
m0n0wall, and testing various configurations, using a cable modem and
a dsl connection in my home for several months now.  The cable modem
connection provides the IP address via DHCP, and tends to change the IP
only when the DHCP client is offline for a few days.  The DSL provides
the IP address via PPPoE, and they like the change the IP address within
a 2 to 4 week period of time.

Pre-1.0 release, I had an IPSec tunnel working between the cable modem
and the DSL.  I even upgraded the 4801 on the cable modem connection over
the VPN connection.  This worked great, and the VPN kept working after
the upgrade.  The DSL end was a 4511-20, and wouldn't upgrade due to
insufficient memory.  I downloaded the configuration, and plugged in the
test 4801 as a replacement for the 4511, and restored the configuration
to it.  Everything seemed to be fine.  I don't remember if I tested the
IPSec tunnel at that point or not.

I later unconfigured that IPSec tunnel while I was trying to configure
IPSec to a FreeBSD machine I have at a colo facility.  I was never
successful getting that to work, and sent an email to this list asking
for help but never got any helpful response.

Fast forward to this weekend:

I'm helping someone set up a network that is connected via a fractional T1
that is shared with voice traffic for his business.  The ISP is providing
the voice and data via a router device that does it all in one little box.
He wants to host his domains on his own server via this connection.
I've convinced him he needs a firewall with DMZ capabilities, and he's
agreed to let me set up m0n0wall for him.  We decided to use a machine
he had that has a Cyrix C3 processor running at 800MHz with 512M of RAM.
We removed the hard disk and configured it to boot from cdrom and ignore
the floppy for boot.

Here's the WAN network information:

XX.YY.148.216/29 network
XX.YY.148.217   router controlled by ISP
XX.YY.148.218   current linksys NAT box for his LAN usage
XX.YY.148.219   IP he wishes to use for the web server in the DMZ
XX.YY.148.220   IP I'm using for m0n0wall box WAN interface while testing
XX.YY.148.221   proxy arp on m0n0wall for testing
XX.YY.148.222   proxy arp on m0n0wall for testing
XX.YY.148.223   broadcast

I'm currently testing with the m0n0wall box on the .220 IP with plans to
move it to .218 when we replace the linksys box with the m0n0wall box.
There is a hardware device behind the linksys that communicates with a
system in a remote LAN via a proprietary encrypted TCP stream and so we
can't just swap things around arbitrarily.  Any outage of more than 30
seconds will cause alarms to sound all over the place (this hardware
deals with security alarm monitoring equipment).  The source IP of the
traffic from this box must be the .218 address.

He already has all the DNS records pointing to the .219 address for his
web and email services.  These services have been down for several months
since he stopped using another ISP after they were compromised (causing a
loss of his data on their end, but he has backups of his own).  He wants
them all back up by the end of this week.  I don't blame him; he's paying
a lot of money for this fractional T1, and wants to get full benefit from

I first thought that bridging WAN with DMZ and enabling bridge filtering
would do the trick.  I wasn't able to make this work.  Then I tried
using a private network range on the DMZ, but so far that hasn't worked
right either.

m0n0wall WAN interface (vr0) is XX.YY.148.220.  It has a proxy arp for
XX.YY.148.219.  I've verified that this is working using tcpdump on
a machine on a hub between the router and the m0n0wall.  m0n0wall LAN
interface (dc0) is  DHCP is enabled, and dns forwarding
is enabled.  m0n0wall OPT1 aka DMZ interface (vx0) is
Server in DMZ is  My laptop plugged into LAN for testing
always gets (it's the only client on the LAN for now).
My laptop at can access completely with
no filtering apparent as expected.  DMZ can't initiate a connection
to anywhere.  Setting up a NAT inbound connection for port 22 on the
WAN address forwarded to works fine.  Setting up a NAT
inbound connection for port 22 or 80 or anything on the XX.YY.148.219
address (which is configured in the "NAT->Server NAT" section of the GUI)
does not work at all.

I finally got tired of standing in front of the keyboard and monitor of
this setup in a cigarette smoke filled room, and tried to set up a IPSec
tunnel back to my DSL line so I could work from my laptop at home to get
this working.  I followed the same options I did when I first set up the
IPSec tunnel between my cable and dsl connection, with no joy.  I also
set up a temporary inbound nat of port 22 on the WAN address to go to the
DMZ host, and also another rule to permit all DMZ addressess full access
to the LAN addresses (so I can access the m0n0wall web gui remotely).
After having no luck getting the IPSec tunnel going between the T1
connected m0n0wall and my DSL connection, I tried to delete everything to
do with that connection and create a new tunnel between the cable modem
and the dsl modem again.  When that didn't work, I tried cable modem to
T1 machine, again with no luck.  I tried all the different options
available in the GUI, making sure both ends matched good, and still no
luck.  Since this was so trivial the first time I did it, I think
something in 1.0 (which all these machines are using) must be broke for
creating new IPSec tunnels.

This is the IPSec error I keep getting:
    racoon: ERROR: pfkey.c:741:pfkey_timeover(): give up to
    get IPsec-SA due to time up to wait.

I've already spent more than 20 hours trying to get this guy's m0n0wall
set up with LAN, DMZ, and WAN, with no luck at all.  I've also spent many
hours battling this new IPSec tunnel problem.  I've far surpassed my
frustration level, and I'm hoping for some help from the list.  The IPSec
problem is a lower priority for me than the DMZ problem.

The final network design for this guy needs to be like this:

T1------+  router XX.YY.148.217/29 |
        | WAN hub XX.YY.148.216/29  |
            | (vr0)
        | m0n0wall XX.YY.148.218/29 |
        | proxy arp: .219 thru .222 |
            | (dc0)             | (vx0) +---------------------------+
            |                   +-------+ LAN switch |
            |                           +---------------------------+
        | DMZ switch |
        | Server:  |

The hub and switches shown do not have IP addresses, and are unmanaged.

Traffic for XX.YY.148.219 ports 80 and 443 need to go to

Traffic from LAN to DMZ needs to pass unmolested.

DMZ needs DNS service.  It also needs NAT access to the net so that
updates can be installed on it.  It runs RedHat Linux.  Someone else
has the nightmare of maintaining the machine, I'm just in charge of the
network and network security.

If any other information is required, I'm willing to share anything that
won't give away identifying information about the client's company.

Does anyone have any suggestions for how to get this client's network set
up correctly with m0n0wall actually passing traffic through for the proxy
arp addresses?

Thanks in advance,