It certainly sounds like your router is the problem, although I agree it is odd that it targets a single site.

Have you done any sort of port forwarding or added any sort of firewall rules to your...

Hi.

What OS, browser, download acceleration software, etc... are you using?

Somehow, as I read your description I immediately though "the router", but it could just as well be your network...