Open Source Webcrawler?

Reply

Join Date: Oct 2007
Posts: 1
Reputation: SkyRenderX is an unknown quantity at this point 
Solved Threads: 0
SkyRenderX SkyRenderX is offline Offline
Newbie Poster

Open Source Webcrawler?

 
0
  #1
Oct 3rd, 2007
I'm looking for an open source webcrawler which can be easily modified. Is anyone aware of a good open source crawler? I'm just enrolled in a dataminning course and would like to develop an algorithm which categorizes web pages. I need to be able to control which pages are indexed and how those pages are categorized, both based on the page content. Thanks.
Reply With Quote Quick reply to this message  
Reply

This thread is more than three months old.
Perhaps start a new thread instead?
Message:




Views: 3480 | Replies: 0
Thread Tools Search this Thread



Tag cloud for Existing Scripts
About Us | Contact Us | Advertise | DaniWeb | Acceptable Use Policy | RSS Feed

©2003 - 2009 DaniWeb® LLC