Bostjan,

Try a program like MTR or for windows 
http://winmtr.sourceforge.net
So you can see which hop is causing your problems. 

Check also cables and try to ping from the M0n0wall webgui. Look also
for In/out errors and Collisions on the Status: Interfaces page. It
could be a duplex problem.

@ Chris
Happy birthday better late than not at all ;-)

Maarten