okay, so i came up with this idea of making a program to evaluate my code. it's supposed to take the code, compile it, run it on several test cases (given in one file each) and report how the code did (which test cases it passed, which cases it failed...)
so... as i've never tried making any real software, (i'm mostly interested in algorithmsss), i have no idea what to do.. and i know this would be the place to ask for help. now, i don't need any code, i only need advice on how this should be done.. also, would it be better to learn some C# for something like this?

First write a program that compiles another program, probably by calling make.exe, nmake.exe, or some other such program that does command-line compiles on your computer. That will be the easiest part because all your program is doing is apawning another program.

The more dificult part will be running the test cases. Write the tests in an ordinary text file using notepad or some other editor that your program will read. Spawn the test program (the one your program just compiled) and stuff its keyboard buffer with the test information just like someone had typed it. That will be the somewhat challenging part, depending on what operating system the two programs run under. Then of course your program will have to learn how to detect what the test program did with the information.

My guestimate is that it will take you a year or so to write a useful program like that which can be marketed.

http://clusty.com/search?input-form=...+software+test

The more challenging aspects are:
- how to manage interaction
- how to deal with programs which exit with some kind of exception
- how to deal with programs which refuse to exit.

What is the form of the result?
- return status - easy
- output to a file - harder. Do things like "Program started at hh:mm:ss" count as a meaningful difference?
- output to a GUI on screen - tricky.