Most routers list clients. They may be divided into how the clients (computers) connect. Those that use the windows default of DHCP might be listed in one section of the routers configuration page, while other clients (static) may be listed elsewhere. Sometimes wireless and wired clients are separated, and sometimes you can find all clients listed together.
If you are concerned about unauthorized access to your network, you may consider setting up encryption.