Interesting. Let me ask a few more questions and confirm some information.
The router's connection to your modem is always good, but your modem will drop the connection from time to time.
The dropped connection only happens at night or is this the only time that you notice it? If it always happens at night, it's possible that your ISP is resetting your connection. It is common for an ISP to do this nightly if you don't have a static IP connection. I don't know about multiple times each night, but I wouldn't put it past some of the companies that I've worked with.
When one computer reboots, the other computer loses its connection. Does the affected computer lose its network connection completely (you can tell that the connection is lost when a dialog "bubble" will pop up from the system tray to notify you) or just its ability to get to the internet? Does the same thing happen if you reboot the other machine?
Are you connecting to the router wirelessly or with cables?
I couldn't find your router at first. It's actually a BEFSR41, which is the same router I used to use. It's a nice router, and I'm not familiar with any connection dropping issues. That's not to say that there aren't any.
Does your router produce a log? If so, see if you can get some information from the router. Save it as a text file and add it as an attachment with your response.