It's safe to say that it is the power supply and no it is not hard to replace.
There should be ~4 screws in the back holding it to your case. If you remove these you'll be able to just take it right out. (unplug it from your wall first, and then unplug it from all of your devices - motherboard, cd/dvd drives, floppy, hard drive, etc) When you get the new one you can just put it right where the old one was. if you need help selecting one, usually a technician can be helpful in choosing one with enough power and the right size for you. (sizes are pretty standard though) Do not go for anything that expensive unless your computer is like a slick gaming rig. Mine is a rosewill 550 watt that has been running nearly nonstop for a few months now, and it has not hiccuped (at about 55 USD).
When you get it screwed into place, just replace all the power connectors. Straightforward as that my friend.