What exactly is this crawler special about, and why didn;t this take off. Recently Wikia acquired this.
http://grub.org/
Sorry if this is a wrong thread to have posted
shouvik.d
33
Junior Poster
Recommended Answers
Jump to PostGrub is a distributed web crawler. It maintains an index of url's including status information (changed/not-changed/unavailable). The problem is, it uses your cpu and bandwidth and they sell the information to other search engines.
All 3 Replies
cooluks
0
Light Poster
I think it is justs an add-ons... To access wiki...
JoetjeF
5
Junior Poster in Training
Grub is a distributed web crawler. It maintains an index of url's including status information (changed/not-changed/unavailable). The problem is, it uses your cpu and bandwidth and they sell the information to other search engines.
x3mario
34
Posting Whiz
Nice post JoejeF. That's a great tip for newbies.
Be a part of the DaniWeb community
We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.