Threading facilities can be found under Boost.Thread. These are essentially the libraries that will form the basis for the upcoming threading standard libraries in C++0x, so better get used to those functions/classes for now, until the standard ones come out. And they are cross-platform, so it doesn't matter much whether you are using Windows or Linux (except that performance of certain things are different from platform to platform because they have different kernel implementations).
You will find examples, on the link, on how to use mutexes.
Windows does not support POSIX threads (pthreads), if I correctly remember my multi-thread assignments at the unversity.
The "standard" Windows API's don't support POSIX threads and most other POSIX API's, but they are available for VC++, and I have used them extensively in the past for cross-platform development. Unfortunately, the biggest problem is that VC++ is not particularly standards compliant. For ANSI/ISO C++ standards compliance, the GNU compiler suite is still amongst the best. There are proprietary compilers from Intel and others that are as good, or possibly better, but they are quite costly and don't run on all available platforms, unlike GNU.