First you have to decide which computer language you want to learn. There are several languages that you can use to write application programs. If you want to write GUI programs than probably VB.NET is good to start -- you can download a free compiler from Microsoft.
I believe you can get visual studio .Net express edition free also. I seem to have the installation CD lying around my desk.
Anyway, u need to know what u wanna develop first. For web server side application, u can use php, asp, ruby on rails, etc. For desktop application, u can use C/C++, VC++, VB, Java, Delphi, etc. I think VB would be an easier language for a beginner.