There's lots to be said about all of the technologies listed in the replies above. Still, I prefer a simpler approach. Begin with XAMPP and Firefox. XAMPP is a PHP/Apache/mySQL package that works on various platforms. I choose the Firefox browser because there's all sorts of development tools available as addons, including:
- Web Developer
- FireFTP
- Colorzilla
- IE Tab
I prefer hand-coding, since I find it quicker than trying to remove the "fluff" that's added by WYSIWYG editors. I'm migrating to Eclipse since it's available cross-platform (I work on Windows at my job, and Ubuntu at home), and has plugins for the various languages that I use, including PDT for PHP.
A last suggestion I'd have is to focus on PHP 5. Yeah, acceptance has been slow, but I think that might be about to change. Plus, the added OOP support will come in handy.
Just a few thoughts....