I'm curious as to what the best CMS to use for a photography site is...

I've had a potential client approach me about having a website made but they would like to update everything themselves including the images in the galleries. I've used Joomla and Wordpress before and I need to find one for the client that will be really easy for them to update.. Especially the gallery. I've seen slide show modules etc for Joomla and they are still far too complicated for a person with no background in coding what so ever... The CMS also needs to be customizable so that I can make a custom template. HTML, PHP and CSS preferred.

Gallery is well-rated but I don't have any experience with it in terms of ease of use or customization.

A good list of CMS systems can be found here. Many have demos available.

There are a huge number of free wordpress templates available for portfolio/photography sites. Try searching for them using the obvious keywords wordpress photo gallery template. The choice is massive, and it can be worth paying to have the template designed by bit removed, if you have a client paying you, as they might think "Wait a minute, free cms and free template??? why is he charging me?"

Considered using Drupal?
I'm using Drupal for a few weeks now, and getting pretty used to it.
I'm pretty sure they have some modules for displaying photo galleries..

Use wordpress, you can get tons of free photogallery themes.