Is there anything out there that can check multiple URL's for errors?

[markybthedaddy]

I am a web marketer, I'm worried that one of the websites I do work for could have links to error pages that are getting clicked on and accruing cost for my customer with no return.

Is there anything out there that could check URL's on mass as I have about 8000 to check and don't fancy doing it one at a time?

Thanks

[The Dude]

Welcome to daniweb :)

This might help you!!!

http://validator.w3.org/checklink

Good luck!

[markybthedaddy]

Welcome to daniweb :)

This might help you!!!

http://validator.w3.org/checklink

Good luck!

Thanks, I tried it but it seems to be only one URL at a time, and it kicked up 59 errors on the homepage alone so I think it is checking the validity of the html and not the validity of the link itself.

[Troy]

I'm sure there are some more commercial products with wizard-type interfaces to help you, but I don't know what they are. I do know that if you are geeky enough, you could build a script that incorporates Wget to find this out.

Wget is a non-interactive command-line utility that can retrieve web content (http/https/ftp).
http://www.gnu.org/software/wget/

In it's simplest form:
wget http://www.google.com
This will retrieve the Google home page and save it to a file in the local directory.

Wget has a recursive download option. You can specify the "depth" level of links to retrieve.
http://www.gnu.org/software/wget/manual/html_node/Recursive-Download.html#Recursive-Download

You can query the status of each request to see if you get 200 or something else--like a 404, etc.

So it's not exactly out-of-the-box plug & play for you, but a solution could be built around Wget.

[ericstenson]

Hi, I can do it with a program... I have my programs do it all the time, to make sure "image links" i use on websites are valid, otherwise i fill in the image with a generic image. if you still need help, stenson1 at gmail dot com.