Get a port scanning tool and install it on your windows computer. I use angry IP scanner (http://download.com.com/3000-2085-10219888.html?tag=lst-0-1) .
Make it scan your network through or through
It will show your 'live' IP addresses and your 'dead' IP addresses on your local network. Then, after you've identified which one your router is...you can input it into your web browser and config it.