[ previous ] [ next ] [ threads ]
 
 From:  Alex Neuman van der Hans <alex at nkpanama dot com>
 Cc:  m0n0wall at lists dot m0n0 dot ch
 Subject:  Re: [m0n0wall] Post mortem autopsy on a dead monowall
 Date:  Tue, 14 Nov 2006 14:02:23 -0500
Looks like that machine is (has been) an open socks4 proxy in South Korea.

Do you have any mail servers behind the monowall? Could be a spammer 
trying to DOS your server.

Kasper Pedersen wrote:
> Synopsis:
>  One of my monowalls crashed twice. I replaced it with a virtualized 
> monowall, and it crashed too.
>  I then dumped the entire physical memory to disk and had a look at it.
>  Strange things were found lurking in the dump.
> 
> 
> And the longer version:
> 
> With two crashes on the same otherwise-reliable monowall in 24 hours, I 
> switched it to a virtualized one, wondering if my hardware (LEX embedded 
> board) had gone bad. Half a day later the VM'ed mono became unresponsive 
> and consumed 100% CPU. I froze the VM (and I still have the frozen one), 
> copied the memory image to a Linux box, and used strings and grep to 
> find the log: On the first two crashes on real hardware the syslog 
> showed nothing, so I worked from the idea that it might be outputting an 
> error message, but didn't live long enough to get it sent off to the 
> syslog server.
> 
> The result of 'strings freebsd.vmem|grep "Nov 14">f2.txt' is at 
> http://n1.taur.dk/deadmono/f2.txt
> Here's the interesting part, the first 5 entries are purely for continuity:
> ----------------excerpt-----------------
> Nov 14 02:47:25 virtualmono ipmon[80]: 02:47:24.990317 lnc0 @0:22 b 
> 192.168.10.17,59838 -> 192.168.10.255,3052 PR udp len 20 547 IN
> Nov 14 02:47:49 virtualmono ipmon[80]: 02:47:48.308027 lnc0 @0:22 b 
> 192.168.10.245,137 -> 192.168.10.255,137 PR udp len 20 78 IN
> Nov 14 02:47:49 virtualmono ipmon[80]: 02:47:48.317109 lnc0 @0:22 b 
> 192.168.10.6,137 -> 192.168.10.255,137 PR udp len 20 78 IN
> Nov 14 02:47:49 virtualmono ipmon[80]: 02:47:48.667763 lnc0 @0:22 b 
> 192.168.10.242,137 -> 192.168.10.255,137 PR udp len 20 78 IN
> Nov 14 02:47:51 virtualmono ipmon[80]: 02:47:50.568596 lnc0 @0:22 b 
> 192.168.10.17,59838 -> 192.168.10.255,3052 PR udp len 20 576 IN
> Nov 14 02:47:54 virtualmono ipmon[80]: 02:47:53.950887 lnc1 @0:23 b 
> 211.205.0.60 -> 80.163.183.190 PR icmp len 20 56 icmp unreach/port for 
> 80.163.183.190,30573 - 211.205.0.60,1026 PR udp len 20 514 IN
> Nov 14 02:47:54 virtualmono ipmon[80]: 02:47:53.951917 lnc1 @0:23 b 
> 211.205.0.58 -> 80.163.183.190 PR icmp len 20 56 icmp unreach/port for 
> 80.163.183.190,30571 - 211.205.0.58,1026 PR udp len 20 514 IN
> Nov 14 02:47:54 virtualmono ipmon[80]: 02:47:53.952054 lnc1 @0:23 b 
> 211.205.0.61 -> 80.163.183.190 PR icmp len 20 176 icmp unreach/port for 
> 80.163.183.190,30574 - 211.205.0.61,1026 PR udp len 20 514 IN
> Nov 14 02:47:54 virtualmono ipmon[80]: 02:47:53.979465 lnc1 @0:23 b 
> 211.205.0.55 -> 80.163.183.190 PR icmp len 20 56 icmp unreach/port for 
> 80.163.183.190,30568 - 211.205.0.55,1026 PR udp len 20 514 IN
> --------------end excerpt----------------
> According to VMWare's log, it died 02:48:02. The windows box is about 4 
> seconds fast, and it takes VMWare a few seconds to notice it, so that 
> timing is about right.
> The new thing in this regard is the last 4 lines. Four address in the 
> same range, ICMP destination unreachable, even though there was no 
> outgoing traffic. At this point I thought it to be a reflection scan, 
> but the box is not suitable for that. And one of the packets has a funny 
> size.
> 
> Since I had failed to sniff the external interface I kludged up a 
> program to search the dump for IP packets where the destination address 
> matched the external interface, and this is what I found:
> (the output is at http://n1.taur.dk/deadmono/packetsearch.txt and 
> contains a 1500 byte dump for each packet)
> This is raw data, prefixed with source IP for readability
> ------------------
> Some TCP
> 219.95.230.76  45 00 00 30 52 D9 40 00 70 06 ED E0 DB 5F E6 4C 50 A3 B7 BE
> 219.95.230.76  45 00 00 30 53 31 40 00 70 06 ED 88 DB 5F E6 4C 50 A3 B7 BE
> 219.95.230.76  45 00 00 30 53 7F 40 00 70 06 ED 3A DB 5F E6 4C 50 A3 B7 BE
> 201.215.230.149  45 00 00 30 98 83 00 00 75 06 F4 75 C9 D7 E6 95 50 A3 
> B7 BE
> 201.215.230.149  45 00 00 30 99 4A 00 00 75 06 F3 AE C9 D7 E6 95 50 A3 
> B7 BE
> 201.215.230.149  45 00 00 30 9A D8 00 00 75 06 F2 20 C9 D7 E6 95 50 A3 
> B7 BE
> icmp echo's (ping)
> 83.190.157.119  45 00 00 3D 6F A8 00 00 76 01 DB 80 53 BE 9D 77 50 A3 B7 BE
> 83.190.157.119  45 00 00 3D 71 2B 00 00 76 01 D9 FD 53 BE 9D 77 50 A3 B7 BE
> this one looks corrupted, protocol 0xB1 is unknown.
> 85.140.42.212  46 D5 04 00 44 00 00 00 80 B1 63 C1 55 8C 2A D4 50 A3 B7 BE
> 
> Now comes the fun ones (if you can read IP headers):
> 
> 211.205.0.66  45 C0 1E 02 F1 F4 00 00 35 01 B4 B9 D3 CD 00 42 50 A3 B7 BE
> 03 01 92 27 00 00 00 00 45 00 02 02 F7 0F 00 00
> This is an icmp host unreachable. But, it is 7682 byte fragmented 
> unreachable. And the IP header is a little different.
> The IP address of this packet did not make it into the log.
> 
> 211.205.0.60  45 00 38 00 3C D5 00 00 75 01 2C 85 D3 CD 00 3C 50 A3 B7 BE
> 03 03 7F 9F 00 00 00 00 45 00 02 02 E0 0C 00 00
> And a 14k port unreachable
> 
> 211.205.0.50  45 C0 1E 02 27 81 00 00 35 01 7F 3D D3 CD 00 32 50 A3 B7 BE
> 03 01 92 37 00 00 00 00 45 00 02 02 5F 8E 00 00
> another 7682 byte host unreach.
> The IP address of this packet did not make it into the log.
> 
> 211.205.0.50  45 C0 1E 02 27 81 00 00 35 01 7F 3D D3 CD 00 32 50 A3 B7 BE
> 03 01 92 37 00 00 00 00 45 00 02 02 5F 8E 00 00
> This one is also not in the log and 7682
> 
> 211.205.0.66  45 C0 1E 02 F1 F4 00 00 35 01 B4 B9 D3 CD 00 42 50 A3 B7 BE
> 03 01 92 27 00 00 00 00 45 00 02 02 F7 0F 00 00
> 
> 211.205.0.58  45 00 38 00 AF 11 00 00 35 01 FA 4A D3 CD 00 3A 50 A3 B7 BE
> 03 03 7F A1 00 00 00 00 45 00 02 02 A2 40 00 00
> 
> this one says 16 bytes extra in the header and 2.5k in size
> 222.134.119.52  49 03 0A 00 44 00 00 00 80 F9 63 C1 DE 86 77 34 50 A3 B7 BE
> C0 A8 97 02 0B 8B 00 19 00 19 02 00 0C 00 00 00
> 
> Another 14k - it appears like an ordinary unreach, but length bytes are 
> swapped.
> 211.205.0.60  45 00 38 00 3C D5 00 00 75 01 2C 85 D3 CD 00 3C 50 A3 B7 BE
> 03 03 7F 9F 00 00 00 00 45 00 02 02 E0 0C 00 00
> 
> This one is gigantic
> 211.205.0.61  45 00 B0 00 2C 89 00 00 75 01 3C 58 D3 CD 00 3D 50 A3 B7 BE
> 03 03 6C DF 00 00 00 00 45 00 02 02 0B 1F 00 00
> 
> and a 14k one.
> 211.205.0.55  45 00 38 00 64 76 00 00 75 01 04 E9 D3 CD 00 37 50 A3 B7 BE
> 03 03 7F A4 00 00 00 00 45 00 02 02 0A F6 00 00
> ---------------------------
> 
> At this point I must admit to lack of clue, so unless someone else goes 
> 'Aha!', my next approach, when the flu I've caught has passed, is to 
> find/make a tool to send strange unreachables.
> 
> Full memory dump: http://n1.taur.dk/deadmono/freebsd.vmem.gz
> 
> /Kasper Pedersen
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: m0n0wall dash unsubscribe at lists dot m0n0 dot ch
> For additional commands, e-mail: m0n0wall dash help at lists dot m0n0 dot ch