Hello! First time Poster :)
I'm trying to create a Java Web Crawler. What I want it to do is pass the crawler a starting site, a max depth/levels of sites to go to (as opposed to number of pages) and save files like images and documents.
The problem is, i'm unsure of how to do this >_<
I believe I know how to process the page to get to a link, but I'm having trouble adding the sites to a queue, and then looking through the queue, parsing those websites and/or even saving the files/documents that I want.
Are there any concrete examples or guides I could use to help me create something like this that will fulfill my needs?
Thank you all for takin the time to read my post :]