I would like to make a kind of archiver that can uncompress rar files. This project will evolve into a program that will guess the password until it unlocks the file. How would I be able to unzip the file using java, and submit a password if it asks for one?

Do you have the format for the contents of a .rar file?
Given that you should be able to extract its contents.

I have just a rar file. I wouldn't know the contents. Currently I have a txt file in it. I don't know how I would be able to extract it. could you please give a link, or an example?

Sorry, I do not know anything about the structure of a rar file. You will have to find that out if you want to write a data extraction program.

My program would be a password finding program for a rar file. I don't actually wan't to decrypt the file.

I have no idea what you mean.
Would your program also find passwords for txt files or images?

here is how it would go. lets say you try to extract the file using my program (using a built in process, or a third party code) then if it needs a password my program will output a form like

The archive has a password:

How many characters: __ (enter number of characters)
[]other symbols (!,.$#, etc)

[cancel] [find password]

then if you click find password then it goes through all of the possible combinations of what you selected, and the number of characters (brute force) to see if it can find a password.

if it needs a password

How do you determine if it needs a password if you are executing a third party program?