I think you might've downloaded something for linux, or maybe the source code. There's a link here http://www.php.net/downloads.php that says "PHP 4.3.7 installer" under "Windows Binaries" - download that. I assume you're using windows because you mentioned ".exe." Once you click on that link, you can download the exe from any one of the mirrors. You'll need a web server before installation. Also, you'll need a database server, like MySQL, if you want to setup a forum on your computer.

If you just want a free forum, register for a free one here: http://invisionfree.com/