<cmath> is the c standard math library. You would need to included it when you want to use the standard math function. Here is a list of all the functions contained in <cmath>. The header has also been updated with c++ functions parameters/returns that are not available in c.
<iostream> and <cmath> are the headers for two different parts of the standard C++ library. The <iostream> library contains the basic input and output classes and objects, including cin and cout. You include it whenever you want to work with I/O streams.
The <cmath> library, as the name implies, contains a variety of mathematical functions and constants, such as fabs() (floating point absolute value) and cos() (cosine). It is an extended version of the library inherited from the C language, hence the 'c' at the beginning of the header name. A full listing of the functions in that library can be found here. You would include it whenever you want to use one of those functions.