What you are looking for is called Internet Connection Sharing by Microsoft and requires that you have two network connections (one wired and one wireless for example).
In windows 7 you got to Network and sharing center then click on Local Internet Connection (or what ever your primary network device is called) and Windows will bring up the Local Internet Connection Status diaog box. Click Properties on the bottom left. When the Local Internet Connection Properties Dialog box comes up click the right tab at the top labeled Sharing. You will see the option to allow other users to connect to the internet through this computer.
Once you turn that option on it will set up so other computers can connect using your computer as the router.
Is this where your PC is using your cell phone's data connection? If so, then tethering works for one PC. For others you need to use the phone as a WiFi access point, or "hot spot" in other terms. If you can tether to the phone, then it should be able to become a hot spot, though bear in mind that usually for phones (such as my Android phone), there is a limit like 4-5 connections. I have used mine for this purpose on numerous occasions, as well as tethering.
However, if you want to physically tether it to one PC, and let others access the connection via either wireless or physical connections, more work is involved, such as setting up a router, a WiFi access point (if you want wireless connections), etc. So, until we know exactly what you are trying to do (your post wasn't clear enough), then this is the best advice I can provide.