Is there anything out there that can check multiple URL's for errors?

Reply

Join Date: Dec 2007
Posts: 2
Reputation: markybthedaddy is an unknown quantity at this point 
Solved Threads: 0
markybthedaddy markybthedaddy is offline Offline
Newbie Poster

Is there anything out there that can check multiple URL's for errors?

 
0
  #1
Dec 13th, 2007
I am a web marketer, I'm worried that one of the websites I do work for could have links to error pages that are getting clicked on and accruing cost for my customer with no return.

Is there anything out there that could check URL's on mass as I have about 8000 to check and don't fancy doing it one at a time?

Thanks
Reply With Quote Quick reply to this message  
Join Date: Dec 2005
Posts: 3,244
Reputation: The Dude will become famous soon enough The Dude will become famous soon enough 
Solved Threads: 26
The Dude's Avatar
The Dude The Dude is offline Offline
Nearly a Senior Poster

Re: Is there anything out there that can check multiple URL's for errors?

 
0
  #2
Dec 14th, 2007
Welcome to daniweb

This might help you!!!

http://validator.w3.org/checklink

Good luck!
Reply With Quote Quick reply to this message  
Join Date: Dec 2007
Posts: 2
Reputation: markybthedaddy is an unknown quantity at this point 
Solved Threads: 0
markybthedaddy markybthedaddy is offline Offline
Newbie Poster

Re: Is there anything out there that can check multiple URL's for errors?

 
0
  #3
Dec 14th, 2007
Originally Posted by The Dude View Post
Welcome to daniweb

This might help you!!!

http://validator.w3.org/checklink

Good luck!
Thanks, I tried it but it seems to be only one URL at a time, and it kicked up 59 errors on the homepage alone so I think it is checking the validity of the html and not the validity of the link itself.
Reply With Quote Quick reply to this message  
Join Date: Jun 2005
Posts: 354
Reputation: Troy is an unknown quantity at this point 
Solved Threads: 5
Troy's Avatar
Troy Troy is offline Offline
Posting Whiz

Re: Is there anything out there that can check multiple URL's for errors?

 
0
  #4
Dec 16th, 2007
I'm sure there are some more commercial products with wizard-type interfaces to help you, but I don't know what they are. I do know that if you are geeky enough, you could build a script that incorporates Wget to find this out.

Wget is a non-interactive command-line utility that can retrieve web content (http/https/ftp).
http://www.gnu.org/software/wget/

In it's simplest form:
wget http://www.google.com
This will retrieve the Google home page and save it to a file in the local directory.

Wget has a recursive download option. You can specify the "depth" level of links to retrieve.
http://www.gnu.org/software/wget/man...rsive-Download

You can query the status of each request to see if you get 200 or something else--like a 404, etc.

So it's not exactly out-of-the-box plug & play for you, but a solution could be built around Wget.
Troy Wolf is the author of SnippetEdit. "Website editing as easy as it gets." IX Web Hosting
Reply With Quote Quick reply to this message  
Join Date: Dec 2007
Posts: 291
Reputation: ericstenson is an unknown quantity at this point 
Solved Threads: 29
Team Colleague
ericstenson's Avatar
ericstenson ericstenson is offline Offline
Posting Whiz in Training

Re: Is there anything out there that can check multiple URL's for errors?

 
0
  #5
Jan 5th, 2008
Hi, I can do it with a program... I have my programs do it all the time, to make sure "image links" i use on websites are valid, otherwise i fill in the image with a generic image. if you still need help, stenson1 at gmail dot com.
--
"Dummy."
Reply With Quote Quick reply to this message  
Reply

This thread is more than three months old.
Perhaps start a new thread instead?
Message:



Similar Threads
Other Threads in the IT Professionals' Lounge Forum
Thread Tools Search this Thread



About Us | Contact Us | Advertise | DaniWeb | Acceptable Use Policy | RSS Feed

©2003 - 2009 DaniWeb® LLC